Notice Me SenpAI

Welcome to Memetic Warfare.

My name is Ari Ben Am, and I’m the founder of Telemetry Data Labs - a Telegram search engine and analytics platform available at Telemetryapp.io. I also provide training, consulting and research so if you have any specific needs - feel free to reach out on LinkedIn.

Let’s start off with a historic event in the field: OpenAI’s first counter covert influence operations report.

Notice Me Sama-pai

OpenAI pulled a Chad move and shadow-dropped its first counter covert influence operation report, surprisingly (to me) written by Ben Nimmo. This is an interesting turn of events and it shows some foresight in the influence space.

I give them a lot of credit for adopting a minimalist style. They’re really embracing the empty space and I’m all for it.

Let’s also just BLUF this bad boy right now:

The report covers primarily already-known operations, such as Doppelganger, Spamouflage, the STOIC operation and even the IUVM, a real throwback. As such, I won’t be going in-depth into every specific case, as frankly a lot of it is what you’d expect (which is fine, it’s the first one). We’ll settle for some specific highlights.

The report showcases the utilization of OpenAI models in all of the above operations:

The gamut is run here - comment generation, generating and notably proofreading articles, and seemingly more. Volume doesn’t equal success though.

LLMs are a full-service solution for IO, as some of the operations used them to create the “appearance of engagement” by spamming comments.

Productivity boosting was also interesting - good on the operators for increasing shareholder value here and making numbers go up. In all seriousness, the sentiment analysis is interesting, but what really hit home was the use of converting double quotes - who here hasn’t struggled with defanging URLs and so on? Relatable.

Not all is easily automated though, people will always make mistakes, everyone has those days:

Let’s move on to the operations. The Telegram-centric Bad Grammar Russian operation is of note.

Overall interesting! It continues though, and attempted to promote anti-Ukraine talking points in English:

All that’s left to bring up is the importance warm water port in Galveston!

Let’s share some other interesting points from the other operations:

Spamouflage used OpenAI to debug code on WordPress, which is frankly always a good idea.

Spamouflage is out there wildin’ as well:

Covering new topics like the public trip of the Chinese Minister of Public Security to Uzbekistan, researching camera lenses, studying Communist doctrine - what can’t you use ChatGPT for?

This is also a good opportunity to bring up some additional commentary. Thomas Rid brought up an excellent point about detection rates and the increasing use of various models:

There’s no doubt that OpenAI reigns supreme currently, and that they’d be the best-suited to identify their own handiwork. Having said that, what does it mean when threat actors eventually purposefully hone their craft, as Rid states, and intermix various models? Will detection rates be accurate with smaller and more diverse/purposefully mixed datasets?

We’ll conclude with Ben Nimmo’s thread on Twitter about this for those who want it straight from the horse’s mouth.

I’ll conclude with a final thought - OpenAI has no lack of capital and means, and they seem to be taking this issue seriously.

The extent to which they become the next hub for trust and safety/counter-IO/cyber operations data is of course not yet understood, but they may be the next big thing. Considering the direction that it seems Meta is headed in, this may be needed.

On that note, let’s move to Meta’s quarterly Adversarial Threat Report.

Throw me the QAT (quarterly adversarial threat report)

Meta’s Q1 2024 Threat Report is now finally available. I had originally planned on starting off this post with the Meta report, but OpenAI left me no choice here, so we started with them.

To be honest, I found this report, similarly to the last one, a bit underwhelming. It seems that Meta is taking more and more time to put out less-detailed reporting than in the past. Just look at the table of contents: a lot of reporting and networks (comparatively, arguably not cumulatively for a whole quarter), but each one gets 1-2 pages with no screenshots or primary information.

I‘m saddened by this, as Meta is arguably the only platform still providing insightful and actionable insight into cyber/IO activity on their platform, and that internal information is critical to the broader space. Hopefully things will pick up i the future.

On that note, let’s begin with a review of my thoughts:

The first point brought up is the use of Gen AI by adversaries. So far the threat is pervasive but seemingly under control, and I’d agree with this assessment, especially as detection methods improve.

Low to medium-level threats can be managed, especially when what we still see is primarily the exploitation of commercial Western tools.

Not to say that Gen AI threats aren’t real - they are - but they can be manageable with technology. This problem will be exacerbated whenever adversaries get around to truly training their own massive models to which Western firms don’t have access, which could make training defensive models very difficult.

From here, the China report was of interest. Originating in China and targeting global Sikh communities, this one raises some questions:

This could be an attempt by Chinese actors to foment unrest in India in a general sense, with the added bonus of targeting the Sikh diaspora. It could also be something else entirely, and unfortunately we don’t have enough information here to make other claims.

Another point - Telegram is present in most of these networks, and increasingly plays a key role. I’m confident that this trend will continue, making tools like telemetryapp.io critical to counter-IO teams.

Meta was also scooped by Recorded Future on the “Emerald Divide” network, and it would’ve been nice to see them refer to the RF reporting. I instead had to go down to the list of threat indicators and check manually to confirm that they are the same networks.

As an aside, my previous hypothesis, in which platform counter-IO and cyber teams would become THE key players in in-depth research on IO, appears to have been wrong. It seems that third-party/non-platform affiliated teams, be they from companies like Recorded Future, Graphika or others, will lead in this space in terms of deep investigation and reporting.

This may be, at least partially, due to the above dynamic of multiplatform networks and messaging applications making single platforms, however important they may be, less critical.

Meta also claims that their enforcement methods work (who knows), which may also contribute to this, but I’m less convinced, and that’s not the Meta team’s fault per se. Meta is massive and sprawling, and it’d be impossible for anyone or any team to close out any exploitation hermetically.

There are some other networks - an Israeli marketing firm, one in Croatia and more, but the one that stood out was the “Unknown origin” network.

These are some active Moldovans and Transnistrians. I’m not one to jump to conclusions, but presumably this is a pro-Russian/Russian-directed effort based on the support for Shor, anti-EU activity and so on. I’d have to look into Russian stances on Madagascar to see if those match up as well.

The report concludes with a long section on Doppelganger, which I’ll skip as it’s a bit long, but take a look if you aren’t sick of Doppelganger coverage yet. They also give themselves multiple taps on the back, which may be partially deserved but as others (and even here on this blog) have shown, it’s far too easy still to find DoppelGanger content on Meta:

Well, that’s the report. Let’s see how the next quarterly report is in approximately 4-6 months, my fingers are fully torqued.

Let’s wrap up with a brief comparison here. The contrast between the two reports is telling, and it seems that OpenAI is capitalizing (rightfully so) on being proactive on IO.

I’m optimistic that we at least have a few years ahead of us of solid reporting from them, so I’m happy. I hope that I could feel the same way about Meta, and they may turn it around (and I really, really hope that they do), but I’m not holding my breath.

The final note really is the centrality of Telegram here. Messaging applications such as Telegram are the present and future of this space, as well as cyber and really any other field.

If you’ve made it this far, thanks for reading. Feel free to share, comment, criticize or whatever, and check out Telemetryapp.io.