How to Fix Java Security Error: “Unsigned Application Requesting Unrestricted Access to System” When Using Dell iDRAC Virtual Console


4 views

When attempting to access Dell's iDRAC virtual console via Java Web Start, many administrators encounter this security roadblock. The core issue stems from modern Java security policies rejecting legacy signing algorithms, while enterprise hardware like Dell servers still relies on them for KVM functionality.

The two key error messages reveal distinct but related problems:

1. "Unsigned application requesting unrestricted access to system" 
2. "Signed with weak signature algorithm MD5withRSA"

This occurs because:

  • Java 7u51+ and Java 8 enforce stricter cryptographic standards
  • The avctKVM.jar uses MD5withRSA which is now disabled by default
  • iDRAC firmware often ships with outdated Java components

Method 1: Java Security Policy Modification (Permanent Fix)

Edit java.security (location varies by OS):

# Windows typically at:
%JAVA_HOME%\jre\lib\security\java.security

# Linux/Mac at:
$(dirname $(readlink -f $(which java)))/../lib/security/java.security

Find and modify these lines:

# Original restrictive setting:
jdk.jar.disabledAlgorithms=MD2, MD5, RSA keySize < 1024

# Modified permissive version:
jdk.jar.disabledAlgorithms=MD2, RSA keySize < 512

Method 2: Temporary Workaround via Command Line

For one-time execution without permanent policy changes:

javaws -J-Djava.security.properties=/path/to/custom.security \
       -J-Dcom.sun.jnlp.assumeAllPermissions=true \
       https://your-idrac-ip:443/console.jnlp

Where custom.security contains:

jdk.certpath.disabledAlgorithms=MD2
jdk.jar.disabledAlgorithms=MD2

For managed environments where policy changes are restricted:

  • Create a dedicated Java runtime for iDRAC access
  • Use Java 8u251 or earlier (before stricter crypto enforcement)
  • Consider the Dell Offline Virtual Console as an alternative

After making changes:

  1. Clear Java cache completely
  2. Restart browser/Java Web Start
  3. Verify using:
keytool -verify -keystore NONE -jarfile avctKVM.jar

Be aware that relaxing these security settings:

  • Makes your system vulnerable to MD5 collision attacks
  • Should only be done for trusted intranet devices
  • Consider firewall rules limiting iDRAC access

For organizations that can't modify Java policies:

  • Use the HTML5 console (iDRAC7 firmware 2.63.60.62+)
  • Configure SSH/IPMI for command-line management
  • Request updated signed binaries from Dell support

When attempting to launch Dell's iDRAC Virtual Console through Java Web Start, many administrators encounter two critical security errors:

Unsigned application requesting unrestricted access to system
The following resource is signed with a weak signature algorithm MD5withRSA

Since Java 8u101, Oracle implemented stricter security policies that:

  • Treat MD5-signed certificates as unsigned
  • Require stronger encryption than the legacy 1024-bit RSA keys
  • Block applications requesting <all-permissions/> without proper signing

Option 1: Temporary Security Adjustment

Edit java.security (location varies by OS):

# For Java 8:
jdk.jar.disabledAlgorithms=MD2, RSA keySize < 1024

# For Java 9+:
jdk.security.legacyAlgorithms=MD5, MD2, RSA keySize < 1024

Option 2: Use Dell's Native Console

Download the standalone version from Dell's support site:

wget https://downloads.dell.com/software/idrac/avctKVM.jar
java -jar avctKVM.jar ip=192.168.1.100

For enterprise environments, create a custom security policy:

keytool -importcert -file idrac.cer -keystore /path/to/cacerts \
-alias idrac -storepass changeit

Then create a .policy file:

grant signedBy "idrac" {
    permission java.security.AllPermission;
};

For headless servers, consider these SSH-based solutions:

# IPMI reset:
ipmitool -H 192.168.1.100 -U root -P calvin chassis power reset

# Dell-specific:
racadm -r 192.168.1.100 -u root -p calvin serveraction powercycle