CVE-2011-3544

What is Oracle Java SE?

Oracle Java SE (Standard Edition) includes the Java Runtime Environment (JRE), which allows web browsers to run Java applets — small programs embedded in web pages. At its peak adoption around 2010–2012, the Java browser plugin was installed on hundreds of millions of machines and was one of the most common components on enterprise desktops. The JRE sandbox (Security Manager) was supposed to prevent applets from accessing the underlying operating system, but sandbox escape vulnerabilities made Java applets the dominant drive-by download vector during this period.

Overview

CVE-2011-3544 is an access control vulnerability in the Applet Rhino Script Engine component of Oracle Java SE. The flaw allows an untrusted Java applet running inside the browser sandbox to invoke privileged Java APIs through the Rhino JavaScript engine, bypassing the Security Manager and achieving arbitrary code execution on the host system.

With a CVSS score of 9.8 and no authentication or user interaction required beyond visiting a web page, this vulnerability was rapidly weaponized by crimeware exploit kits and became one of the most widely exploited Java flaws of 2011–2012.

Oracle patched CVE-2011-3544 in Java 7 Update 1 and Java 6 Update 29, released October 18, 2011.

Affected Versions

Technical Details

Java applets run within the JRE's Security Manager sandbox, which is supposed to block access to sensitive APIs (file system, network, process execution). The Rhino engine is a JavaScript interpreter integrated into Java's scripting framework (javax.script), allowing Java applets to execute JavaScript.

The vulnerability was an access control flaw (not a memory corruption issue): the Rhino engine allowed applet code to call certain Java methods that were supposed to be restricted by the Security Manager. By constructing specific JavaScript sequences within an applet, an attacker could invoke privileged Java APIs to escape the sandbox entirely and execute arbitrary code as the user running the browser.

Why this was so exploitable:

• No memory corruption required — just API calls, making the exploit reliable across all JVM architectures and OS versions
• The CVSS score of 9.8 (no auth, no user interaction beyond visiting a page) reflects that a silent drive-by download required nothing from the victim
• The exploit could be embedded in a single web page and served to any visitor with a vulnerable Java plugin

Discovery

The vulnerability was identified through Oracle's internal security processes and coordinated with security researchers. The October 2011 Oracle Critical Patch Update addressed this flaw alongside related Java applet sandbox issues.

Exploitation Context

CVE-2011-3544 was rapidly integrated into crimeware exploit kits, most notably Blackhole Exploit Kit — the dominant drive-by download platform of 2011–2013. Blackhole operators used this vulnerability alongside browser and plugin exploits to silently install malware on any visitor to a compromised or malicious website with a vulnerable Java installation.

Drive-by campaigns using CVE-2011-3544 delivered banking trojans, spam botnets, and ransomware precursors to millions of victims globally. The Java browser plugin's combination of near-universal deployment and persistent sandboxing vulnerabilities made it the #1 drive-by attack surface throughout this era.

Remediation

1. Apply Oracle CPU October 2011 — update to Java 7u1 / Java 6u29 / Java 5u33 immediately
2. Disable the Java browser plugin if not strictly required — the browser plugin has been the source of the vast majority of Java CVEs; Java SE itself (for server and desktop applications) is much less exposed
3. For Java 6 and 7: both versions have since reached end-of-life (Java 6 EOL: 2013, Java 7 EOL: 2015) — migrate to Java 17 LTS or Java 21 LTS
4. In enterprise environments, use Group Policy or endpoint management to control which browsers/sites are permitted to run Java applets
5. If any legacy application still requires the Java browser plugin, isolate it in a dedicated, network-segmented VM that is not used for general browsing or email