When using Apache 2.2.3 (Debian) as a reverse proxy with mod_proxy
to forward requests to Jetty 6.1.18 on port 8080, you encounter 502 errors specifically for long-running requests (several minutes). Direct requests to Jetty work fine, indicating the issue lies in the proxy configuration.
# Current ineffective settings you've tried:
ProxyTimeout 600
KeepAliveTimeout 600
retry=1 acquire=3000 timeout=600
The debug log shows the proxy connection is terminated prematurely:
[error] proxy: error reading status line from remote server
[error] proxy: Error reading from remote server returned by /
For Apache 2.2.x, these parameters need special attention:
# In httpd.conf or virtual host configuration:
ProxyPass / http://backend:8080/ \
timeout=600 \
retry=0 \
connectiontimeout=300 \
acquire=3000 \
keepalive=On
# Essential mod_proxy settings:
ProxyTimeout 1200
ProxyBadHeader Ignore
ProxyIOBufferSize 32768
Jetty 6.1.x has its own timeout defaults that may conflict with Apache:
# In jetty.xml (adjust these values):
<Set name="maxIdleTime">7200000</Set> # 2 hours
<Set name="soLingerTime">-1</Set>
<Set name="requestHeaderSize">8192</Set>
Check TCP-level timeouts that might affect the connection:
# Linux system settings to verify:
sysctl net.ipv4.tcp_keepalive_time
sysctl net.ipv4.tcp_keepalive_intvl
sysctl net.ipv4.tcp_keepalive_probes
For extremely long operations, consider this event-based approach:
<Location /long-process>
ProxyPass http://jetty:8080/long-process flushpackets=on
ProxyPassReverse http://jetty:8080/long-process
SetEnv proxy-sendchunks 1
</Location>
After applying changes, monitor with:
tail -f /var/log/apache2/error_log | grep -E 'proxy|timeout'
netstat -tn | grep 8080 # Verify connection states
Remember to restart both Apache and Jetty after configuration changes:
service apache2 restart
service jetty restart
When working with Apache 2.2.3 (Debian) as a reverse proxy for Jetty 6.1.18, developers often encounter the frustrating 502 error with the message "Error reading from remote server". The issue manifests specifically during long-running requests (typically several minutes), while shorter requests complete successfully.
From the described scenario, we can identify several important characteristics:
- Direct requests to Jetty (port 8080) work correctly
- The problem only occurs through Apache's mod_proxy
- KeepAlive adjustments didn't resolve the issue
- The error suggests a connection timeout between Apache and Jetty
The current Apache configuration shows several timeout-related settings that need optimization:
ProxyTimeout 600
KeepAliveTimeout 600
ProxyPass / http://www.mydomain.fi:8080/ retry=1 acquire=3000 timeout=600
While these values appear reasonable, there are additional factors to consider in the proxy chain.
Here's an enhanced VirtualHost configuration that addresses multiple potential failure points:
<VirtualHost _default_:80>
ServerName www.mydomain.fi
# Connection and timeout settings
Timeout 1200
ProxyTimeout 1200
ProxyBadHeader Ignore
# Proxy configuration
ProxyRequests Off
ProxyVia On
ProxyPreserveHost On
# Enhanced ProxyPass with additional parameters
ProxyPass / http://localhost:8080/ \
retry=5 \
acquire=5000 \
timeout=1200 \
keepalive=On \
connectiontimeout=1200 \
flushpackets=auto
ProxyPassReverse / http://localhost:8080/
# Error handling
ProxyErrorOverride Off
# Connection pooling
<Proxy *>
ProxySet connectiontimeout=1200
ProxySet timeout=1200
ProxySet acquire=5000
ProxySet retry=5
</Proxy>
</VirtualHost>
Simultaneously, ensure your Jetty configuration has matching timeout values in jetty.xml:
<Call name="addConnector">
<Arg>
<New class="org.mortbay.jetty.nio.SelectChannelConnector">
<Set name="host"><SystemProperty name="jetty.host"/></Set>
<Set name="port">8080</Set>
<Set name="maxIdleTime">7200000</Set> <!-- 2 hours -->
<Set name="lowResourcesConnections">5000</Set>
<Set name="statsOn">false</Set>
</New>
</Arg>
</Call>
Implement logging to identify exactly when the connection fails:
# In Apache configuration
LogLevel debug proxy:trace5
# Sample log format
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \
%{ms}Tms %Dµs" proxy_format
CustomLog /var/log/apache2/proxy_debug.log proxy_format
For extremely long-running processes, consider these architectural alternatives:
- Implement asynchronous processing with status polling
- Use web sockets for long-lived connections
- Break large operations into smaller chunks
Remember to test configuration changes with:
apachectl configtest
service apache2 reload