Unplugging: Identifying and countering misinformation and radicalisation on social media

The incredibly individualised experience of social media, despite its omni-present reach has proved the ideal habitat for misinformation to thrive. This misinformation does not exist without intention or purpose, often it serves to undermine specific efforts to dismantle exploitative power structures or produce individual profit. A culture of misinformation and distrust creates a pipeline to radicalisation that results in both passive and active violence,

with radicalisation and disinformation online resulting in extremist violence, and attacks on democratic processes. At the same time, in the face of a global public health crisis disinformation undermines attempts to create a culture of collective good that is necessary for protecting communities and saving lives. Conspiracy theories are not new to the modern age but the specific conditions of social media and online communities is such that conspiracy, distrust and misinformation have been elevated to the mainstream. To counter misinformation we must begin to understand what it looks like, where it comes from and the effect it has on people's lives.

Abuse of power comes as no surprise. Conspiracy and corruption do exist in the world, and social media can be a powerful platform to encourage users to think critically and empathetically about the structures that define us, and the power and authority that lies behind these structures. However a certain kind of radicalisation that fosters distrust, tension and division is fraught among online communities. Despite its omnipresent connection to the world, the internet is an isolating place.The radicalised whose faith is forged in conspiracy operate on a deep suspension of reality.

Often at the heart of these radicalising conspiracy theories is the belief in some form of deliberate agenda that seeks to control the lives and freedoms of the every-man. They are highly critical of authoritative sources and the mainstream media (MSM), and see every criticism of their beliefs and most revered figures as evidence of this global conspiracy. Missteps, false predictions and behaviours that can’t be justified within their worldview are written up as ‘false flags’ - that is orchestrated efforts by their enemies to undermine their spread of the Truth. While COVID conspiracy takes on many different forms, such as an outright denial of the severity of the virus to the belief that it is a biochemical attack from either a foreign state or profit driven pharmaceutical company, the core belief system remains: that your way of life is threatened, that you’re not being told the truth, and you must do everything you can to counter these malicious forces.

The far right conspiracy movement Qanon is a quintessential example of this belief system and a prominent source of disinformation and an increasing culture of distrust. Qanon originated on 4chan with an anonymous user, known as ‘Q’, who claimed to be a US military official with high level clearance and insight into the ‘secret war’ that president Trump was waging against the ‘shadow government’ - a Kabal of satan worshiping pedophiles that controlled the government, business and media. The absurdity of these claims have not stopped the meteoric rise of Q’s following and the assimilation of these beliefs into the wider public conversation.

The tenets of Qanon are familiar, they follow the thinking of typical anti-Semitic conspiracies and foster a culture of white-supremacy. The distrust they hold in MSM is echoed in Trump’s frequent denunciations of fake news and his belief that the liberal media was orchestrating a staged campaign against him. The particular danger of Qanon is that the nature of social media results in the promotion and dissemination of Qanon obsessions without real awareness of the source. The beliefs of Qanon might seem so far-fetched that it can be difficult to conceive of Q’s followers having a meaningful impact on society, and yet without believing in, or even being aware of Qanon many have adopted a worldview that is rooted in conspiracy, and peddle Q’s delusions without consideration or reflection.

Despite its obsession with corruption, when it comes to the figures that the conspiracy seeks to rever, any evidence of corruption is denied, ignored or explained away by the ‘False Flag’ belief. Much of Qanons belief system depends upon a distrust and denunciation of MSM. This distrust is not wholly un-founded, there is power in owning the news and the potential for corruption and mal-intent is huge. Conspiracy theories thrive where there is existing inequality and political tension, they are a way of addressing these concerns while maintaining an existing belief system and a means of avoiding addressing structural concerns. For example, Qanon has a largely pro-Trump base who believe that MSM ran a co-ordinated and extensive attack on Trump throughout his campaign and presidency. Alternatively, critics of Trump argue that the media's incessant obsession and coverage of Trump broadcast his delusions to a mainstream audience, and was instrumental in his ascension to power. Notably, at the time of Trump's campaign his future son in law and Senior-Advisor to the President Jared Kushner owned the New York Observer. Kushner’s pipeline from Newspaper Mogul to political figure shows firsthand the power that lies behind news publishing, yet acknowledging this would mean admitting to Trump's own role in corruption. In mind bending feats of confirmation bias, any structural analysis of corruption is denied in order to hold on to the fundamental worldview. Explanations of corruption that involve structural analysis of power are dismissed out of hand and used themselves as evidence of the conspiracy theory.

Unlike traditional extremists who have increasingly used social media as a form of recruitment but whose dominant presence has been on the streets, Qanon exists primarily online and has thrived among social media where algorithms suggest increasingly incendiary content to users. A users journey to radicalisation may begin with somewhat sympathetic content; many of Qanon’s most prominent conspiracies focus around child abuse. For example the ‘Wayfair’ conspiracy theory that originated on reddit this July and was spread rapidly by Qanon supporters, suggested that Wayfair, a furniture retailer, was using it’s site as a front for child trafficking, the evidence being that certain items were listed for between $12,699.99 and $14,499.99 and were named for people, ie Yaritza, Alivya, and Samiyah. In reality the listings were made by AI and an error had caused them to generate implausible prices, and naming items after people is a common trend across retailers. Despite the easily explainable justifications for the ‘evidence’, the conspiracy picked up massive traction on Twitter and Tiktok - according to Darren Linvill, an associate professor studying social media disinformation, on Twitter alone the conspiracy garnered 1.8 million total mentions, with 600,000 in a 24-hour period.

This explosion of content exposes a much wider audience to the views and beliefs of conspiracy movements without any real understanding of the actors driving this content or their motivations. A focus on child abuse is common among white supremacist groups who make unsubstantiated claims about the danger ‘our’ or ‘your’ children face. These groups know their most violent views are not palatable to a wider audience so dress their prejudices up in a more ‘sympathetic’ cause - the nature of the claim is highly emotional (who wouldn’t care about ‘the children'?) and thus more likely to be shared and less likely to be questioned. It also paints a backdrop of a world of deliberate evil and sows the seeds for a kind of anxiety that primes people for radicalisation.

Social media allows posters to more easily disguise their intentions, and users are less likely to recognise information as biased or coming from an illegitimate source on social media. Generally speaking, most people are apt at identifying the biases within a source when they know to look out for that bias, however social media's status as a leisure space which users inhabit in order to unwind, relax or take a break means they are letting their guard down when interacting with their feeds. Users expect to see posts from family and friends, people with whom they have some amount of familiarity or respect. This means that posts promoted through algorithms that come from external connections such as suggested posts or the likes and interactions of connections are not evaluated with the same rigor that a person might approach something they believe to be a new-source. Everything becomes anecdotal information - someone might scroll past misinformation without much consideration, priming them for acceptance of more outlandish suggestions.

When exposed to these initial conspiracies a user may want to find out more, or seek out groups who share similar concerns. Once a part of an online community or group radicalisation escalates. The user's feed becomes an echo chamber of confirmation and members of the group escalate and reinforce each other's beliefs, often under the guise of debate, “It looks like a discussion but really it’s just a lot of people nodding their heads”. Online communities also encourage increasing isolation and distance from reality. The Community encourages the individual to identify fully with the group and it’s belief systems, fostering a stigmatisation of external beliefs which results in distancing from friends and family members. Often friends or family members find it necessary to estrange themselves, finding their loved one increasingly disconnected from reality, more paranoid and incendiary and has difficulty de-escalating their beliefs. As the victim of radicalisation becomes increasingly isolated it becomes easier to maintain their suspension of reality, they are less likely to encounter beliefs or information that counters their own world view. Eventually the sense of personhood and community becomes so deeply entwined with the belief system that deprogramming seems impossible. Where it is attempted the process of deprogramming can take years and often proves a far too emotionally draining and complex feat for individuals to undertake with their own radicalised loved ones.

“It’s so easy to jump from these groups to other groups because of Facebook’s algorithms, because you have some concerns about 5G and then it recommends an anti-Bill Gates group and you go ‘oh well I’ve heard some bad things about Bill Gates, maybe I should join that and do some more research’ then it recommends an anti-vaccine group and that recommends you a group about the great awakening, which is Qanon. So you very quickly and efficiently have radicalised yourself into a violent anti-Semitic far right conspiracy cult without having any intention of doing so.” - Micheal Rothchild, The Wellness to Qanon Pipeline, Maintenance Phase

Existing online communities that are predisposed to distrust of traditional authorities are far more likely to be susceptible to conspiracy theories and disinformation, making them a target for some degree of radicalisation. For example, online wellness communities have a longstanding distrust of conventional medicine - miracle cures for all manner of ailments, afflictions and illness are commonplace despite any meaningful evidence surrounding their efficacy. Belief in a more traditional or holistic approach to well being is often harmless, and there are many reasons a person's hesitancy toward centralised medical care might be sympathetic. The danger arises with the surrounding culture, which fosters a distrust of centralised expertise and promotes the belief that authoritative sources are keeping something from you. To promote their miracle cures, grifters often tout solutions that ‘Doctors don’t want you to know about’, capitalising off of and exacerbating the distrust of central authority. Again the belief that there is a Truth that is being deliberately hidden from you is central to the promotion of alternative authorities such as online communities, reactionaries and conspiracy theorists. This kind of exposure to distrust of conventional medicine paves the way for anti-vax sentiment and COVID denial.

Deradicalisation is a difficult and involved process. If you see a loved one begin down this path it is important to maintain a neutral form of communication with them. Interference in their beliefs may only feed the persecution complex that drives these communities and cause them to further isolate themselves from the external world. By maintaining a connection that focuses on shared experiences and understandings you maintain a lifeline to reality. A believer has to want to get out to be able to free themselves from the grip of radicalisation but once someone has wholly isolated themselves from friends and family it may be difficult for them to give up the only community they have left.

On a wide scale the only sustainable solution to disinformation and radicalisation is prevention and early interference. ‘Combating misinformation online: re-imagining social media for policy-making’ suggests that social media is currently used by policy makers simply to inform, where it could be used to engage and connect with citizens and monitor disinformation. Throughout the pandemic whole swathes of the population turned to social media as a means of accessing information and staying connected with the world, while social media companies struggled to address the demand for reliable and readable information. Sites such as YouTube and Twitter have made high profile changes to address the spread of misinformation and prioritise reliable sources. However the nature of these sites requires constant engagement from the user base, thus prioritising incendiary and emotive content irregardless of legitimacy. While social media companies have a responsibility to monitor incorrect information, as long as there is uncertainty alternative explanations will arise. It is essential to prioritise the clear and concise delivery of legitimate information and understand the fears and anxieties shared among communities in order to address them.

Until policy-making catches up to the complexities of the internet age, it falls on the individual user to monitor their own engagement with information on social media. Everyone is susceptible to disinformation, particularly when they are not expecting it. It’s essential to be able to identify the intention of the content you are consuming and what can be achieved by any information or content you may choose to share or engage with.