Understand the alert
The portcheck_connection_timeouts alert calculates the average ratio of connection timeouts when trying to connect to a TCP endpoint over the last 5 minutes. If you receive this alert, it means that the monitored TCP endpoint is unreachable, potentially due to networking issues or an overloaded host/service.
This alert triggers a warning state when the ratio of timeouts is between 10-40% and a critical state if the ratio is greater than 40%.
Troubleshoot the alert
Check the network connectivity
- Use the
pingcommand to check network connectivity between your system and the monitored TCP endpoint.ping <tcp_endpoint_ip>If the connectivity is intermittent or not established, it indicates network issues. Reach out to your network administrator for assistance.
- Use the
Check the status of the monitored TCP service
- Identify the service running on the monitored TCP endpoint by checking the port number.
- Use the
netstatcommand to check the service status:
netstat -tnlp | grep <port_number>
If the service is not running or unresponsive, restart the service or investigate further into the application logs for any issues.
Verify the load on the TCP endpoint host
- Connect to the host and analyze its resource consumption (CPU, memory, disk I/O, and network bandwidth) with tools like
top,vmstat,iostat, andiftop. - Identify resource-consuming processes or applications and apply corrective measures (kill/restart the process, allocate more resources, etc.).
- Connect to the host and analyze its resource consumption (CPU, memory, disk I/O, and network bandwidth) with tools like
Examine the firewall rules and security groups
- Ensure that there are no blocking rules or security groups for your incoming connections to the TCP endpoint.
- If required, update the rules or create new allow rules for the required ports and IP addresses.
Check the Netdata configuration
- Review the Netdata configuration file
/etc/netdata/netdata.confto ensure theportcheckplugin settings are correctly configured for monitoring the TCP endpoint. - If necessary, update and restart the Netdata Agent.
- Review the Netdata configuration file