The error message clearly indicates an SSL handshake failure despite having -Dcom.sun.management.jmxremote.ssl=false in your JVM parameters. This paradox occurs because Java's JMX/RMI implementation still attempts SSL negotiation even when explicitly disabled.

-Dcom.sun.management.jmxremote.port=8084
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
# Critical additions:
-Dcom.sun.management.jmxremote.rmi.port=8084
-Djava.rmi.server.hostname=your.private.ip.address

With dual NICs (public/private), Java may select the wrong interface for RMI communication. The RMI registry port (typically random) needs explicit binding:

# Verify ports are listening correctly:
sudo netstat -tulnp | grep java
# Should show both 8084 and the RMI port

# IPTables rules example:
iptables -I INPUT -p tcp --dport 8084 -j ACCEPT
iptables -I INPUT -p tcp --dport 30000:40000 -j ACCEPT  # For dynamic RMI ports

Your FQDN must resolve correctly on both client and server. Test with:

nslookup your.server.fqdn
ping your.server.fqdn  # From client machine

# Alternative connection URL format:
service:jmx:rmi://your.private.ip:8084/jndi/rmi://your.private.ip:8084/jmxrmi

For Tomcat deployments, modify setenv.sh:

CATALINA_OPTS="$CATALINA_OPTS \
-Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.port=8084 \
-Dcom.sun.management.jmxremote.rmi.port=8084 \
-Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.authenticate=false \
-Djava.rmi.server.hostname=$(hostname -i)"

As an alternative to VisualVM, try connecting with JConsole first:

jconsole your.server.fqdn:8084

# If connection fails, enable verbose logging:
java -Djavax.net.debug=all -jar jconsole.jar

Since you're using private DNS and NICs, ensure:

• Routing tables prefer the private interface
• No reverse path filtering is blocking traffic
• TCP keepalive is enabled (add -Dsun.rmi.transport.tcp.responseTimeout=60000)

Here's a verified configuration that works in similar environments:

# Java startup parameters:
java -Dcom.sun.management.jmxremote \
     -Dcom.sun.management.jmxremote.port=8084 \
     -Dcom.sun.management.jmxremote.rmi.port=8084 \
     -Dcom.sun.management.jmxremote.ssl=false \
     -Dcom.sun.management.jmxremote.authenticate=false \
     -Djava.rmi.server.hostname=192.168.1.100 \  # Explicit private IP
     -Dsun.rmi.transport.tcp.responseTimeout=5000 \
     -Dsun.rmi.transport.proxy.connectTimeout=5000 \
     -jar your_application.jar

# Corresponding connection URL:
service:jmx:rmi:///jndi/rmi://192.168.1.100:8084/jmxrmi

---

When migrating Java applications between Windows and Linux environments, JMX connectivity issues often surface. The "Failed to retrieve RMIServer stub" error with SSL handshake failures suggests a fundamental mismatch in security expectations between client and server.

Your setup shows non-SSL JMX configuration:

-Dcom.sun.management.jmxremote.port=8084 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.authenticate=false

Yet the client (jvisualvm) appears to be attempting SSL negotiation, evident from the stack trace showing SSLHandshakeException.

First verify basic connectivity:

telnet your.fqdn.com 8084
nc -zv your.fqdn.com 8084

Then check if RMI registry is accessible:

curl http://your.fqdn.com:8084/jmxrmi

Option 1: Force plain RMI communication by adding these JVM parameters:

-Dcom.sun.management.jmxremote.rmi.port=8084
-Djava.rmi.server.hostname=your.fqdn.com
-Dcom.sun.management.jmxremote.local.only=false

Option 2: If SSL is required, configure it properly:

-Dcom.sun.management.jmxremote.ssl=true
-Djavax.net.ssl.keyStore=/path/to/keystore
-Djavax.net.ssl.keyStorePassword=changeit
-Djavax.net.ssl.trustStore=/path/to/truststore

For multi-homed hosts (like your dual-NIC setup), explicitly set the binding:

-Djava.rmi.server.hostname=private.nic.ip.address

As an alternative approach, try using jstatd:

jstatd -J-Djava.security.policy=jstatd.all.policy \
       -J-Djava.rmi.server.hostname=your.fqdn.com \
       -p 8084

With policy file jstatd.all.policy containing:

grant codebase "file:${java.home}/../lib/tools.jar" {
    permission java.security.AllPermission;
};

Verify JMX connectivity without GUI tools:

jconsole service:jmx:rmi:///jndi/rmi://your.fqdn.com:8084/jmxrmi

Even though port 8084 is open, RMI uses dynamic ports. You'll need to either:

• Open a range (e.g., 30000-40000)
• Fix RMI ports with: -Dcom.sun.management.jmxremote.port=8084 -Dcom.sun.management.jmxremote.rmi.port=8084