Troubleshoot Splunk Secure Gateway performance issues
If you're experiencing connection issues with Splunk Secure Gateway, see Troubleshoot Secure Gateway Connection Issues.
If you're using a proxy, make sure to open port 443 outbound to prod.spacebridge.spl.mobi to enable Splunk Secure Gateway. See Splunk Secure Gateway requirements for more information.
Troubleshoot error codes
Refer to the following table if you receive an error code in Splunk Secure Gateway:
Error code | Meaning | What to do |
---|---|---|
301 | Invalid Splunk platform instance URL | If you're using another service to reach Spacebridge, make sure your Splunk platform instance URL is correct in the configuration. |
303 | Redirect Error | If you're using another service to reach Spacebridge, make sure your Splunk platform instance URL is correct in the configuration. |
305 | Proxy Required | Make sure your proxy is properly configured. See Use a proxy server with Splunk Secure Gateway to learn how to configure a proxy for Splunk Secure Gateway. |
400 | Unable to complete request | Try again later or contact your Splunk administrator. |
401 | Authentication error | There's an error with the authentication process. Try clearing your cache, or see Troubleshoot registration issues and make sure your credentials are valid. Or Contact Splunk support and see Troubleshoot performance issues to troubleshoot further. |
403 | You don't have the right permissions. | See Configure Splunk Secure Gateway permissions. |
405 | Method not allowed. | Splunk Secure Gateway doesn't accept this request. You can Contact Splunk support. |
404 | Page not found | Make sure the URL is valid and try again. |
406 | Request not acceptable | Splunk Secure Gateway received an unexpected request. Modify your request and try again. |
407 | Proxy authentication required | Authenticate with your proxy and try again. See Use a proxy server with Splunk Secure Gateway for more information about using a proxy with Splunk Secure Gateway |
408 | Request timeout | Your request timed out. Try again. |
409 | Entity already exists | Create a different entity. |
410 | Page no longer exists | The URL no longer exists. Remove references to this URL. |
413 | Request payload is too large | Try the request again with a smaller payload |
429 | You've made too many requests | Splunk Secure Gateway cannot handle that many requests at a time. Lower your number of requests and try again. |
431 | Request header is too large | Make the request header smaller and try again. |
500 | The app is able to make the request to Spacebridge, but it encountered an internal error. | Contact Splunk support and see Troubleshoot performance issues to troubleshoot further. |
502 | Invalid response | Try again or Contact Splunk support. |
503 | The app is unable to connect to Spacebridge. The connection is unavailable or the Splunk Secure Gateway modular inputs aren't working. | Make sure you have internet connection, or Contact Splunk support. |
504 | Server timed out | Try again. |
510 | Not enough information | Provide more information and try again. |
511 | Network authentication required | Authenticate to network connection and try again. |
Troubleshoot registration issues
If you or a Splunk admin changes your Splunk user credentials, your device becomes unregistered.
To register your device again, use Splunk Secure Gateway. An admin must enable token authentication so you can access the Connected Experiences apps again.
If token authentication isn't enabled, users aren't able to access the Connected Experiences apps. Enable token authentication so users can register their devices again after their credentials have been changed.
See Enable token authentication in the Splunk Enterprise Securing the Splunk Platform manual to learn how to enable token authentication. See Register your mobile device to a Splunk platform instance in the Use Splunk Secure Gateway manual for registration steps.
Troubleshoot performance issues
You can use the troubleshooting dashboards in Splunk Secure Gateway to run tests or you can use the command line.
Use the Splunk Secure Gateway troubleshooting dashboards
Splunk Secure Gateway version 2.5.4 and higher comes with troubleshooting dashboards to help identify possible issues. To view the troubleshooting dashboards, navigate to Dashboards > Troubleshooting Dashboards in Splunk Secure Gateway.
Splunk Support might ask you to use the following dashboards during the troubleshooting process.
Troubleshooting dashboard | How to use |
---|---|
Request tracing | You can troubleshoot and trace a particular request. Select a time range and a request ID to get troubleshooting logs and a stack trace. Or, you can get a request ID from the Connected Experiences mobile apps.
See the following documentation to learn how to view logs and get a request ID from the Connected Experiences mobile apps: |
Subscription Tracing | You can troubleshoot and trace a particular panel that isn't loading. Select a time range and a panel ID to get troubleshooting logs and a stack trace. Or, you can get a panel ID from the Connected Experiences mobile apps.
See the following documentation to learn how to view logs and get a request ID from the Connected Experiences mobile apps: |
Single Value Test | A single-value panel that tests if a simple dashboard loads in Splunk Secure Gateway. It helps identify possible causes for issues during troubleshooting. |
End-to-End Websocket Test | Initiate a sample request to run an end-to-end websocket test for Splunk Secure Gateway, device authentication, and Spacebridge messaging. Select Single Request or Subscription Message and identify the request type. This dashboard requires JSON Web Tokens (JWT) to be enabled. |
KV Store Collections Status | If you're running into issues with the Connected Experiences apps, Splunk Support might ask you to use this dashboard to check the KV Store status. This dashboard provides details about the KV Store collections status. |
Use the command line to run a modular input test
To determine if the Splunk Secure Gateway modular inputs are running, run the following command at the command line for your operating system. The following example responses indicate that the modular inputs are running.
Operating System | Command | Response | |
---|---|---|---|
Linux |
$ ps ax | grep modular_input |
25468 ? Ss 0:24 python /opt/splunk/etc/apps/splunk_secure_gateway/bin/ssg_subscription_modular_input.py 26201 ? Ssl 0:28 python /opt/splunk/etc/apps/splunk_secure_gateway/bin/secure_gateway_modular_input.py | |
Windows | index=_internal sourcetype="secure_gateway_app_internal_log" ssg_subscription_modular_input OR secure_gateway_modular_input "Running" |
INFO [secure_gateway_modular_input.app] [secure_gateway_modular_input] [do_run] [938] Running Splunk Secure Gateway modular input on search head, shard_id=splunkenterprise INFO [ssg_subscription_modular_input.app] [ssg_subscription_modular_input] [do_run] [977] Running Subscription Manager modular input on search head |
Disable and re-enable the Splunk Secure Gateway modular inputs
If the Splunk Secure Gateway modular inputs aren't running, there might be a requirements issue. See Get Splunk Secure Gateway to make sure your Splunk platform meets the Splunk Secure Gateway requirements. Or you can disable and re-enable the following Splunk Secure Gateway modular inputs in Splunk Web in Settings > Data Inputs:
- Splunk Secure Gateway
- Splunk Secure Gateway Subscription Processor
- Splunk Secure Gateway Mobile Alerts TTL
- Splunk Secure Gateway Role Based Notification Manager
- Splunk Secure Gateway Metrics Collector
- Splunk Secure Gateway Subscription Clean Up
- Splunk Secure Gateway Registered Users List
- Splunk Secure Gateway Deleting Expired Tokens
- Splunk Secure Gateway Migration Modular Input
Troubleshoot data request issues
See the following suggestions to troubleshoot data request issues.
Unable to see a dashboard or panel
Configure the appropriate dashboard permissions to make dashboards visible in the Connected Experiences apps. See Configure Splunk Secure Gateway and dashboard permissions for more information about configuring dashboard permissions for the Connected Experiences apps. Apps without these permissions don't appear in the Connected Experiences apps.
If you can't find a dashboard you're looking for and you're sure that it contains only supported visualizations, make sure the app that the dashboard comes from is selected in the App Selection tab. See Select which Splunk apps to show dashboards from in the Connected Experiences apps.
If the dashboard you're looking for isn't on the dashboard list or a panel isn't loading, the dashboard may contain an unsupported visualization or configuration. For a list of supported visualizations and dashboard configurations, see Visualization support for the Connected Experiences apps in the Splunk Secure Gateway Release Notes manual.
Unable to see dashboard list
If you can't see your list of dashboards, you might be running into a service connection error. See Troubleshoot Splunk Secure Gateway Connection Issues in the Splunk Secure Gateway Use Splunk Secure Gateway manual.
Unable to send alerts to mobile devices from Splunk Enterprise Security
To send alerts from Splunk Enterprise Security, add splunk_secure_gateway
to the update_es Application Inclusion List field in the App Imports Update page of Splunk Enterprise Security.
See Enable Splunk Enterprise Security mobile alerting for more information.
Expired license error
Splunk Secure Gateway requires a Splunk Cloud or Splunk Enterprise license. You might receive a notice saying that one of your licenses has expired. This notice appears when any of your Splunk licenses has expired. Splunk Secure Gateway functionality might not be affected if your Splunk platform license is still valid.
If you're experiencing issues with Splunk Secure Gateway and receive this notice, make sure your Splunk platform license is still valid.
Troubleshoot Splunk Secure Gateway network connection issues |
This documentation applies to the following versions of Splunk® Secure Gateway: 2.6.3 Cloud only, 2.7.3 Cloud only, 2.7.4, 2.8.4 Cloud only, 2.9.1 Cloud only, 2.9.3 Cloud only, 2.9.4 Cloud only, 3.0.9, 3.1.2 Cloud only, 3.2.0 Cloud only, 3.3.0 Cloud only, 3.4.251, 3.5.15 Cloud only
Feedback submitted, thanks!