This Report Goes to 11

Welcome to Memetic Warfare.

Before starting this week, if you’re interested in Latin America, check out a new Substack from a friend of the blog covering Latin American affairs.

Let’s kick this week off with Google’s latest report on abuse of Gemini, Google’s generative AI platform, available here.

This report is hefty and worth reading in full, but let’s review the highlights anyway as there are some points I want to bring up.

Firstly, as always - key findings:

At first glance these line up with past reporting on abuse of generative AI tooling. The conclusion that LLMs aren’t in fact “breakthrough” tools shouldn’t surprise anyone despite the hype.

The next finding is also noteworthy:

This is more interesting to me as it’s been less-researched. Threat actors prefer to just use simple queries or prompts instead of actually trying to compromise an LLM or exploit a prompt via an attack, which takes a lot of work and effort.

Having said that, there were some low-energy jailbreaking attempts:

Previously available exploits, such as one found on Github, were used:

The next section is the use of Gemini to support malware development, reconnaissance, privilege escalation and other parts of the kill chain:

Iran out there wilding should surprise no one at this point, they’ve shown their willingness to throw caution to the wind and prompt with reckless abandon. North Korea is a refreshing threat actor to find on one of these lists too.

Google found over 20 state APTs using Gemini. Would be nice to get a list of the 20!

The report goes to 11 as far as I’m concerned regarding structure and infographics as well, really great stuff:

Iranian threat actors just can’t get enough of Gemini it seems:

I don’t usually share so many screenshots from a report, but here it just feels right:

Not surprising that content generation is a big one, and interesting also IMO that they even bother using Gemini for actual “research” on general topics, like the Iran-Israel war and general topics that would be better researched manually.

APT 42 was the most prolific, which matches their recent spate of activity targeting the US, Israel and other targets:

From here it’s also cool to see other topics beyond Israel that interest APT 42, such as China’s defense industry.

From here, the report moves on to China, Russia and North Korea.

Chinese actors were up there with Iran in terms of volume:

I personally find it really interesting seeing the mundane tasks that APTs need help completing:

Chinese threat actors also tried to get inside of Gemini’s head:

North Korean APTs used Gemini primarily for their financially-motivated cyber activity:

I like this section on North Korean IT workers and their past use of AI tooling:

Russian APTs seemed to be risk-averse and on the whole avoided Gemini:

The final section focuses on IO specifically. Iran is responsible for 3/4 of all IO prompts, a fascinating statistic showing just how open Iran is to using Western tooling while having negative levels of opsec. Spamouflage/Dragonbridge is also called out as the main Chinese user of Gemini, which isn’t surprising.

Iran is number one here:

There were at least 8 Iranian groups using Gemini, see specific prompts/examples below:

China is also active of course:

Google only identified 3 groups:

Russia opened up here:

They used it for general research, translation, content management etc:

So, quite a bit of interesting stuff here, but overall it bucks the now-thoroughly bucked conventional wisdom of generative AI:

• Generative AI will not fully revolutionize cyber/IO, yet

• Gen AI is more useful for IO activity than most cyber activity, at least as it appears now

• People apparently use Gemini

I’m looking forward to seeing more of these types of reports, and hope that we get them from Anthropic and other foundational LLM companies.

We’ll conclude this week’s post with a recommendation on further reading from Alliance for Europe’s Saman Nazari, available here. Their report covers Doppelganger activity on Bluesky, so if you’re interested - give them a read and a follow.

That’s it for this week!