Troubleshoot the Splunk Add-on for F5 BIG-IP

For helpful troubleshooting tips that you can apply to all add-ons, see Troubleshoot add-ons in Splunk Add-ons. For additional resources, see Support and resource links for add-ons in Splunk Add-ons.

"Destination unreachable" errors

Check that you have opened port 443 or whichever f5 server port you're using in your firewall to enable the communication between F5 BIG IP server and Add-on via the iControl API over SSL.

Errors

You can find most of the runtime errors for the Splunk Add-on for F5 BIG-IP in the $SPLUNK_HOME/var/log/splunk/Splunk_TA_f5_bigip_main.log file or respective input's log file at $SPLUNK_HOME/var/log/splunk/splunk_ta_f5_bigip_input-<input_name>.log. You can find other errors in the $SPLUNK_HOME/var/log/splunk/splunkd.log file. The Splunk Add-on for F5 BigIP uses a checkpoint to store the parameters with which the API Call is done. You must enable the KVStore while performing the data collection.

Change the logging level

You can change the logging level for this add-on by navigating to Configuration > Logging. The default logging level for the add-on is INFO. You need to re-enable the input to reflect the changes made to the log level.

F5 BIG-IP Telemetry Streaming General Troubleshooting Tips

By default, BIG-IP Telemetry Streaming logs to restnoded.log (stored on the BIG-IP at /var/log/restnoded/restnoded.log), at the info level. At the info log level, you can see any errors that BIG-IP Telemetry Streaming encounters. The consumers within BIG-IP Telemetry Streaming also log an error if they are not able to connect to the external system.

Known Limitations

The Splunk Add-on for F5 BIG-IP version 6.10.0 collects the data using Telemetry Streaming from F5 Servers. F5 Telemetry Streaming does not support multiple API calls at the same time. When more than one API call is sent to the Telemetry Endpoint, it returns a "503: Service Unavailable Error". Because of this limitation, note the following when performing the data collection using modinput:

• The user must include all the templates for a particular server in a single input, for which they want to perform the data collection.
• When the user tries to disable/delete the input, the API calls will be made to the Telemetry endpoint so that F5 Server will stop sending the data to the Splunk platform. Because of this, it may take some time to disable/delete the input.
• In that situation, when some of the API calls fail to execute, during the next invocation, the API calls will be made to the Telemetry Endpoint to stop the data collection. After all the API calls are successful, a log will be generated in the $SPLUNK_HOME/var/log/splunk/splunk_ta_f5_bigip_input-<input_name>.log file saying Data Collection is complete for the disabled API Calls. After that, user can enable the input again to perform the data collection. During that time, the input state will display as enabled in the UI.
• When the data collection is in progress, if the user tries to edit/disable/delete the input, an error message communicates that Data Collection is in progress. You can check again after some time.

F5 Telemetry Streaming frequently sends certain internal auditing events, for example:

{"telemetryServiceInfo":{"pollingInterval":300,"cycleStart":"2022-02-18T08:59:38.388Z","cycleEnd":"2022-02-18T08:59:39.300Z"},"telemetryEventCategory":"systemInfo"}