Can a trusted Java Virtual Machine (JVM) interrogate an untrusted JVM? At the application level, a trusted application can request a certificate from an unknown party. Is there something analogous at the JVM level?
Java is compiled into platform-independent byte code, executed by an interpreter called the Java Virtual Machine (JVM). A Web browser loads and runs Java code using either its built-in Java Virtual Machine architecture or a user-installed one. From a security viewpoint, there are two Java applet types: signed and unsigned. Unsigned applets are deemed untrusted. They have no access to the local machine and can only access the server where it’s hosted. A signed applet contains a signature from its creator or publisher that the browser can verify. Once the signature is verified and the computer user agrees, a signed applet can operate more like an ordinary standalone program. The rationale is that the author of the applet is now known, and will be responsible for any deliberate damage.
For either type of applet, an independent JVM instance is created every time you run java.exe. These JVMs are completely separate from each other; you can even run different versions of JVM on the same machine. Because each applet is running in a strictly separated virtual machine, objects in one JVM can't send messages to objects in another JVM or share data with them.
JVMs need to “talk” to each other for various reasons, including accessing information on other JVMs. Two JVMs can only talk to each other using inter-process communication methods, just like two non-JVM processes would do. You can use Java Remote Method Invocation (RMI) in which the methods of remote Java objects can be invoked from other Java virtual machines. However, you need to know and specify the hostname and port of communication of the JVM with which you need to communicate. Another problem you face when wanting your trusted JVM to establish a trust relationship with another untrusted JVM is that untrusted applets are generally restricted from making network connections, starting other programs or making native calls on the client host system.
Consider reading Michael Young’s explanation of how a JVM can accept a website certificate by manually importing it into the JVM's keystore to see if this could be a solution that works for you.
This was first published in October 2011