At first, there was a reason to feel optimistic about Elon Musk’s takeover of Twitter. Mr Musk seemed to be interested in disrupting a cross-firm, collusive, and overly restrictive equilibrium on the standards and norms of social media content moderation. While I understand that the scope and value of “free speech” is always contested, Musk’s claim that he wanted to enable freer speech sounded good to me. As is now clear, Mr Musk did not have the expertise needed to do that. But it’s also clear that an influential group of people, influential enough to threaten him with a CFIUS review, organize a boycott of major advertisers, encourage mass migration to Mastodon, and generate a lot of noise and ridicule on Twitter itself, did not want him to succeed. In fact, it seems like some people would rather destroy Twitter than allow Musk to alter its policies and practices.

So before explaining how Musk’s hasty decisions have undermined Twitter’s systems of identity and reputation and destroyed his own stated agenda, let me explain why the Twitter acquisition was so viciously controversial – and how Twitter could have benefited from a well-managed change in policy. Despite the reams of commentary, that story doesn’t seem to be getting told.

Social media and political polarization

For the past 30 years American politics have become more polarized, and for the last 6 years there has been a power struggle between the “poles” over the control of public discourse. In the past, rightwing and leftwing movements clustered around different mass media channels, catering to audiences over which the media owner could gain some exclusivity. But the rise of social media caused both factions to converge on the same platforms and intermingle. A war over the norms applicable to that space ensued.

The election of Donald Trump – and similar political outcomes in Latin America and Europe (and here I include Russia as part of Europe) – intensified and internationalized the struggle over regulatory and ideological control of social media outlets. Not only do these major US platforms reach global audiences, they have created a new arena for geopolitical competition. The Russian influence operations and the attempted linkage of Trump to Russia/Putin helped to further polarize and securitize political discourse on social media. Trump’s January 6 coup attempt and subsequent riot were clear manifestations of the polarization and mistrust. It led to one of the worst betrayals of the Constitution in all of US history. The U.S. almost failed to pass the most basic test of democratic government: a peaceful transfer of power based on election results.

In the decade preceding this political tornado, the California-based social media platforms (Google/YouTube, Facebook, Instagram, Twitter) were gradually consolidating their holds on the user population. This gave them a powerful ability to intermediate public discussions. Interactive, transnational social media replaced national newspapers and broadcast/cable programming networks as the primary locus of public discourse and influence. As this happened, the political value of regulating social media increased. Social media, after all, not only enables high-scale and relatively uncontrolled social exchanges, it can also systemically restrict access and regulate the intermediation of these exchanges to enforce norms.

Just as Trump sought to discredit critical media with the “fake news” label and associate the platforms with the “woke culture” of the left, so the left, seeking to explain (away) the rise of the Right, started claiming that social media was mainly responsible for rightwing movements, extremism and mass shootings. This claim has no basis in social science but signaled to anyone paying attention that content moderation policies were being drawn into the political polarization process. YouTube, WhatsApp, and Facebook were now seen as breeding grounds for political opposition movements that the left found threatening. Extending platform content regulation deeper into political communication was seen as the answer.

Far from de-fanging the polarized and conflict-prone discourse, however, this battle to control content on platforms amplified and widened the population’s ideological differences. The GOP-led right and the Democratic Party-led left competed for control of the media, in the process sorting themselves into well-identified, manipulable factions.

Demands to regulate social media content now came from both sides. Conservatives wanted to regulate them to correct “political bias,” proposing, among other things, laws to subject them to common carrier regulation. They also wanted to get rid of Section 230, which shields the platforms from legal liability for user statements. The right’s attack on Section 230 was purely punitive: it would actually reinforce restrictions on speech by generating a high risk of lawsuits, and encourage platforms to suppress even more.

The left, on the other hand, wanted the platforms themselves to do more and more content suppression. Even as they accused platforms of being the economic equivalent of slot machines, (a habit-forming vice that requires therapeutic intervention by political doctors), they also demanded that they take more responsibility for what we can say and do online. The rise of a nationalist and culturally conservative right, epitomized by the victories of Trump and Bolsonaro, made many transnational human rights advocates question or even abandon their commitment to free speech, as they began to equate it with power inequalities and the growth of rightwing groups and violence. Differing approaches to Covid and vaccination further widened the chasm between left and right messaging. The Left began to advocate for the suppression of any speech that clashed with their norms and push for a standard of allowable speech based not on individual rights or free political competition, but on more paternalistic social protection standards. Social media, they said, must be managed to degrade or shut off the circulation of “hate speech,” “disinformation,” “misinformation,” “conspiracies,” unscientific claims about vaccines, climate and gender, and programmed to suppress any organized movements generating such messages.

The major platforms gradually conformed to the demands for tacitly partisan content moderation. Initial attempts to avoid active intervention by the platform kings were subjected to intense political pressure: congressional hearings, the Haugen exposures, lawsuits, threats of new legislation. The platforms were headquartered in the bluest of blue states, California, as was much of the Internet infrastructure. Though each platform made independent decisions and there were minor differences among them (Twitter actually being relatively permissive), over time the obvious cultural and political inclinations of the platforms’ owners, employees and ideological milieu made conformity the path of least resistance. The platforms developed numerous coordination mechanisms, often global in scope, which homogenized their approach to content control. Trust and Safety evolved into a recognized community with its own professional association, establishing a stable hegemony over norms.

And so the Democrats won that battle. The lies, absurd conspiracies and links to violent actors on the Trumpian right made it much easier for this to happen. Trump was banished from Twitter, his most extreme allies denied hosting services, domain names, distribution via the app store duopoly, or DDoS protection. Advocates on the left advocated silencing certain kinds of groups and messages associated with Trump’s base. The partisan slant of this equilibrium was confirmed by the shadow ban on the Hunter Biden laptop story in the midst of the 2020 election.

Overreach and Musk’s entry

The institutionalization of more active content moderation had many beneficial achievements. It made the platforms think more carefully about how to handle virality and engagement. It reduced obvious forms of harassment and incitement; it targeted coordinated inauthentic behavior by nation-state propagandists; it tagged labels onto the spokespersons for authoritarian states.

Inevitably, however, there was overreach. Legitimate disagreements that needed to be debated were tagged as “misinformation.” Many commentators on the left began to advocate expanding surveillance and content regulation to private, two-way media such as Telegram, WhatsApp, iMessage. Their argument was always the same: these unregulated spaces were being used by the rightwingers who had been purged from more public spaces, so these private spaces, too, should be subject to surveillance and “moderation” so as to suppress “dangerous” voices and movements. The fact that point-to-point messaging services such as the telephone system had been content-neutral common carriers for more than a century did not matter to them. The idea that clashing viewpoints could circulate and that people could mute, reject or refute bad ideas was abandoned. There was no middle ground, they implied, between aggressive, progressive-norm-dominated content moderation and no content moderation at all – any loosening of controls would make social media a “hellscape” and the polity would soon descend into fascism, (because, apparently, fascist appeals always win in an open public forum). Even voices that were not extremist, but willing to stray outside the Overton Window that the left wanted to enforce, like Joe Rogan, were attacked. Videos were taken down, tweets retracted, programs threatened with boycotts. Musk’s intervention was seemingly motivated by disgust with this.

Now it should be clear why Musk’s takeover of Twitter kicked off such a storm. It threatened to disrupt the inter-firm cooperation and normative hegemony shared by all the major California-based platforms and their audiences. It disrupted the narrative that “there is no alternative.” The prospect of normative competition in this space scared some people and made others very angry. And Twitter, it turns out, was heavily populated, and discursively dominated, by people who were deeply committed to the norms and policies he was threatening.

Musk’s Mistakes

Some breath of fresh air, some diversity and competition in this space was long overdue. Musk’s management missteps, however, have totally derailed his challenge. As a result, we are not having a constructive discussion of or experiment with a more liberal content moderation policy, but a chaotic ruckus.

  • He started laying off everyone in sight to reduce costs before he had made any viable adjustments in the business model and operational requirements
  • He made the transformation of Twitter into a public performance by engaging in petty levels of twitter-tit-for-tat with his admirers and detractors, inviting antagonism and burdening the transition process with half-baked ideas and distracting flare-ups.
  • He told his content moderation department’s public spokesperson to tweet that the layoffs did not affect Twitter’s content moderation capability, and then a few days later laid off the person who made the announcement (a confidence-destroying move).
  • Most damaging of all, he destroyed Twitter’s reputation signals. The other problems were distracting, but not fatal. This one shot himself in the foot – or possibly the head. It couldn’t have played into his enemies’ agenda more completely.

From Musk to Mush

Apparently, Musk did not understand the importance of Twitter’s identifier system. Nominally committed to eliminating bots (automated accounts under a fake identity) and making Twitter “the world’s most reliable and accurate source of information,” Musk ended up undermining one of the primary methods Twitter users had for assessing the reliability and accuracy of the sources of tweets.

According to (pre-Musk) Twitter, “The blue Verified badge on Twitter lets people know that an account of public interest is authentic.” Prior to Musk, Twitter bestowed these symbols at its own discretion, via a not very transparent process. But while doled out too stingily, they did effectively differentiate famous people and brands from ordinary users and fly-by-night accounts.

Hardly a week into the transition, Musk decided to sell “Verification” subscriptions as one possible way to replace advertising as a major revenue source. The whole idea emerged hastily around October 31 from a set of casual probes circulated on Twitter by Musk himself. As Julian Sanchez pointed out, Musk didn’t think through the economic incentives generated by making a reputation symbol a “consumption good.” Just as Twitter was having its identity-policing crisis, here was Musk proposing to sell verification of the authenticity of identifiers to anyone, for a trivial price. It’s as if he was thinking, as he was cracking down on Musk imitators, “how can I make the impersonation problem worse?”

Get worse it did. As the new verification subscriptions were implemented, all kinds of flagrantly fraudulent accounts were set up, and deliberately misleading messages sent out from them. It became clear that “verification” was simply an $8 transaction; there was no process to ensure that the claimed identities were valid. Was it possible that the trimmed-down staff didn’t have the capacity to vet them? There was a gray symbol and a blue symbol and before anyone could figure out what they meant they were proven to be meaningless and the gray one withdrawn.

My favorite is this exchange about Chiquita, the banana company with a…history.

This was already a major partisan battle. Musk’s mistakes, however, played into the hands of his opponents. As his privacy lawyers quit, The Federal Trade Commission warned him that “Our revised content order gives us new tools to ensure compliance, and we are prepared to use them.” A Democratic Party Congressman, who worked with a Washington Post reporter to “verify” a fake account in his name, threatened Musk: “Fix your companies. Or Congress will.” It is no accident that all of these threats (and nearly all of the others) are coming from Democrats.

What about Trump?

Two weeks ago, it seemed to us that the biggest potential crisis facing Musk was whether to re-admit Donald Trump to Twitter. Perhaps the only saving grace of the current crises is that the significance of that decision has been drastically diminished. Not only did Trump’s political significance fade because of the midterm election results, but Twitter is now surrounded by so much turmoil that adding Trump into the mix would have a lot less impact. Heck, we wouldn’t even know for sure if it really was Trump!

While we hope that Twitter will not reinstate Trump, not yet, it is healthy to force people to think about the possibility. Just as Twitter had every right (under Section 230 and the U.S. Constitution) to ban Trump, so the current regime has every right to reinstate him. (We are sure that all the anti-Trumpers will consistently uphold that position. Cough.) The significance of the ban can be overstated. Trump has many ways of getting publicity aside from Twitter, and he uses them. The idea that there is some magic in tweets that makes his messages irresistible is just a thin rationalization for suppression. The election results from 2018 to 2022 have proven that Trump is a liability to the Republican Party and that open debate between his supporters and detractors can produce rational results in most cases.

The only good reason not to reinstate Trump is his adherence to the stolen election lie. Anyone who insists on sticking to that story is no better than an impersonated account. If (and only if) he will publicly disavow his claim that the election was stolen, I would be ok with letting him back on.