Troubleshoot Splunk Secure Gateway network connection issues
If you're experiencing network connection issues, make sure that your Splunk Secure Gateway version is up to date and configure your network settings appropriately.
To check for network connection, you can try the following troubleshooting steps:
- Check for network connetion
- Test for wss connection
- Make sure you've configured your proxy correctly
- Search the Splunk Secure Gateway logs for errors
Check for network connection
The Splunk Secure Gateway app comes with a Secure Gateway Status dashboard that displays connection status, KV Store status, message requests, and more. Green color status indicates a connected state. View the Secure Gateway Status dashboard in the Splunk Secure Gateway Dashboards tab.
If Spacebridge is connected, the status dashboard looks like this:
Or, you can manually check for connection by doing the following tests.
Verify the search head host has access to Spacebridge. Run the following command:
$ curl https://prod.spacebridge.spl.mobi/health_check
This response verifies that the search head host has access to Spacebridge:
Spacebridge Status: OK
If you're using a Windows system that does not include the curl command, type https://prod.spacebridge.spl.mobi/health_check in a web browser.
If you don't receive a
Spacebridge Status: OK response when checking if the search head host has access to Spacebridge, or if the modular inputs aren't running, there might be an installation issue. See Troubleshoot Splunk Secure Gateway performance issues to check whether the modular inputs are running, and see Get Splunk Secure Gateway for installation information.
If Splunk Secure Gateway isn't loading, clear your browser cache, or use an incognito tab. Splunk Secure Gateway might not load because of cache conflicts.
Test for wss connection
Splunk Secure Gateway uses the WebSocket protocol to maintain communication between Spacebridge and your Splunk platform instance. Open port 443 outbound to
prod.spacebridge.spl.mobi to allow the WebSocket connection.
To check if you have WebSocket connection, run the following curl command at the command line:
curl -i -N -H "Connection: Upgrade" -H "Upgrade: websocket" -H "Host: echo.websocket.events" -H "Origin: https://echo.websocket.events" -H "Sec-WebSocket-Key: d3d3LnNwbHVuay5jb20=" -H "Sec-WebSocket-Version: 13" https://echo.websocket.events
The expected result is the following:
HTTP/1.1 101 Switching Protocols Connection: Upgrade Upgrade: websocket Sec-Websocket-Accept: s8sh/L5fxeyDfaeTGQj8NBegoIg= Via: 1.1 vegur �*echo.websocket.events sponsored by Lob.com
Proxies and firewalls might interrupt this connection. Adjust your proxy or firewall by doing the following steps:
- Make sure you're using a compatible proxy server. See Use a proxy server with Splunk Secure Gateway for more information about using a proxy server with Splunk Secure Gateway.
- If your proxy is running SSL decryption, it must support WebSockets or exempt
Make sure you've configured your proxy correctly
- Make sure you're using a compatible proxy. See Use a proxy server with Splunk Secure Gateway for more information about compatible proxies.
- If you're using a supported proxy, ensure that your proxy is acting as a true passthrough proxy and isn't stripping any HTTP headers.
Search the Splunk Secure Gateway logs for errors
Use the Search & Reporting app to search the Splunk Secure Gateway logs for errors.
Search for unusual errors
Search for unusual errors in the Search & Reporting app:
index=_internal source=*secure_gateway* ERROR AND NOT SUBSCRIPTION
Trace a specific request
Search for a specific request ID to trace a specific request:
index=_internal source=*secure_gateway* request_id
Then, copy and paste the request ID in the search bar:
index=_internal source=*secure_gateway* request_id=<your_id_here>
Export the logs to further troubleshoot Splunk Secure Gateway. Export Splunk Secure Gateway logs as raw events so that you can use the
"secure_gateway_app_internal_log" source type in your search.
Troubleshoot Splunk Secure Gateway performance issues
This documentation applies to the following versions of Splunk® Secure Gateway: 2.4.0, 2.0.2, 2.5.6 Cloud Only, 2.5.7, 2.6.3 Cloud only, 2.7.3 Cloud only, 2.7.4, 2.8.4 Cloud only, 2.9.1 Cloud only, 2.9.3 Cloud only, 2.9.4 Cloud only, 3.0.9, 3.1.2 Cloud only, 3.2.0 Cloud only, 3.3.0 Cloud only, 3.4.251