Troubleshoot the Splunk Add-on for BMC Remedy
General troubleshooting
For helpful troubleshooting tips that you can apply to all add-ons, see Troubleshoot add-ons in Splunk Add-ons. For additional resources, see Support and resource links for add-ons in Splunk Add-ons.
Find relevant errors
The Splunk Add-on for BMC Remedy contains the following internal logs in the $SPLUNK_HOME/var/log/splunk
directory:
Data source name | Source type | Description | Collection method | API |
---|---|---|---|---|
splunk_ta_remedy_alert.log
|
ta:remedy:alert
|
Logs related to creating or updating incidents in Remedy using Splunk software alert actions. | File Monitoring | None |
splunk_ta_remedy_ticket.log
|
ta:remedy:ticket
|
Logs related to creating or updating incidents in Remedy using Splunk software custom commands. | File Monitoring | None |
splunk_ta_remedy_incident.log
|
ta:remedy:incident
|
Logs related to Splunk software endpoint which fetches incident details from correlation id in Remedy. | File Monitoring | None |
remedy_incident_modalert.log
|
ta:remedy:modalert
|
Audit logs of activities related to Splunk software alert actions. | File Monitoring | None |
splunk_ta_remedy_rest_account_validation.log
|
ta:remedy:rest:account_validation
|
Logs related to Server Validation using REST on REST Account tab. | File Monitoring | None |
splunk_ta_remedy_soap_account_validation.log
|
ta:remedy:soap:account_validation
|
Logs related to Server Validation using SOAP on SOAP Account tab. | File Monitoring | None |
splunk_ta_remedy_input.log
|
ta:remedy:input
|
Logs related to Data Collection. | File Monitoring | None |
splunk_ta_remedy_data_input_checkpoint.log
|
ta:remedy:data_input:checkpoint
|
Logs related to checkpoint mechanism during data collection. | File Monitoring | None |
Configure the logging verbosity level in the add-on setup page.
Search for the following event types to find errors relevant to the Splunk Add-on for BMC Remedy.
- Search
index=_internal sourcetype=ta:remedy:alert
for errors related to creating or updating incidents in Remedy using alert actions. - Search
index=_internal sourcetype=ta:remedy:ticket
for errors related to creating or updating incidents in Remedy using commands. - Search
index=_internal sourcetype=ta:remedy:incident
for errors related to the Splunk platform endpoint which fetches incident details from the correlation id in Remedy. - Search
index=_internal sourcetype=ta:remedy:modalert
for errors related to the execution of alert actions. - Search
index=_internal sourcetype=ta:remedy:rest:account_validation
for errors related to saving account on REST Account Tab. - Search
index=_internal sourcetype=ta:remedy:soap:account_vlidationticket
for errors related to saving account on SOAP Account Tab. - Search
index=_internal sourcetype=ta:remedy:input
for errors related to data collection for inputs configured. - Search
index=_internal sourcetype=ta:remedy:data_input:checkpoint
for errors related to checkpoint during data collection.
SSLError after add-on upgrade
If you are using a self signed certificate for your remedy server and see the following error in the TA logs of the add-on after upgrading, refer to the steps in the Splunk Add-on for BMC Remedy.
SSLError occurred. If you are using self signed certificate and your certificate is at /etc/ssl/ca-bundle.crt, please refer the troubleshooting section in add-on documentation.
splunk_ta_remedy_settings.conf not replicated to other members in a search head cluster
If you update the required fields in the splunk_ta_remedy_settings.conf
file on a search head member in a search head cluster, the file will not be synced to the other search head members. You must manually update the required fields in the splunk_ta_remedy_settings.conf
file on each search head in the cluster separately.
Custom command errors
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:676)>
If you receive a <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:676)> error, check your add-on's configuration in splunk_ta_remedy_settings.conf
. This issue comes when SSL communication and certificate validation is enabled, but the server certificate cannot be verified from available certificates with the client.
[Errno 2] No such file or directory
If you receive a [Errno 2] No such file or directory error, then you may have specified a certificate path for the ca_certs_path
field in the additional_parameters
stanza of splunk_ta_remedy_settings.conf
. Check if the server's CA certificate is available at the specified location.is you need to disable the certificate validation or add/verify ca cert path as mentioned in Encrypted/Unencrypted Communication Section.
For Splunk Cloud users, the ca_certs_path
located at ca_certs_path = /etc/ssl/certs/ca-certificates.crt
. Location may vary, based on the operating system (OS) used.
IOError:[Errno 13] Permission Denied error
If you receive a IOError:[Errno 13] Permission Denied error, then you need the folder permissions where server ca certificate is present and of local folder of addon where wsdl files from Remedy server will be downloaded.
Server raised fault: 'ERROR (48255): ; The contact for the creation of an Incident cannot be found, please verify the contact information and try again.'
If you receive a Server raised fault: 'ERROR (48255): ; The contact for the creation of an Incident cannot be found, please verify the contact information and try again. error while creating an incident, check whether the account of whose first name and last name are provided in the command has rights to create an incident on the Remedy server.
Server raised fault: 'ERROR (326): Required field cannot be blank.; 2506'
If you receive a Server raised fault: 'ERROR (326): Required field cannot be blank.; 2506 error while creating/updating an incident, check all the mandatory fields required to be passed while creating or updating the incidents on Remedy server.
Server raised fault: 'ERROR (1291018): ; You do not have the authority to modify incident, "INCXXXXX"
If you receive a Server raised fault: 'ERROR (1291018): ; You do not have the authority to modify incident, "INCXXXXX" , check if the provided user is a member or an associate member of either the owner group or the assigned group in Remedy server.
If you receive a Server raised fault: 'ERROR (1291205): ; The Assignee field must contain a value. Choose an assignee from the associated field menu, then save the Incident., verify on your Remedy server that the assignee field is not empty for that specific incident. If it is blank, then add the assignee value.
'Remedy Web Service has not been setup' error
If you receive a Remedy Web Service has not been setup error, check Configuration > Account to verify whether remedy server configuration is configured. If it is blank, then add the valid configuration of your BMC Remedy server.
urlopen error [SSL: UNKNOWN_PROTOCOL] unknown protocol
If you receive a urlopen error [SSL: UNKNOWN_PROTOCOL] unknown protocol error, verify that your Remedy Server URL as mentioned in Configuration > Account is encrypted via HTTPS with a valid SSL certificate.
Add-on using protected URL instead of public URL
If you see that the Splunk Add-on for BMC Remedy is still using protected
in WSDL URL, although the add-on has modified it to public
, check if you have explicitly specified it in the [remedy_ws]
stanza in splunk_ta_remedy_settings.conf
. Modify the splunk_ta_remedy_settings.conf
file, if required.
Alert action errors
Alert action error: No groups were found using automated routing. You need to manually select a group.
If you receive a No groups were found using automated routing. You need to manually select a group error, after configuring an alert action, this issue is due to the configuration in your BMC environment. You need to make sure that routing rules are configured in your BMC Remedy deployment. The assignment routing rules can be configured in your Remedy environment by navigating to the "Administrator Console > Application Administration Console >> Custom Configuration tab" section.
Use the custom Remedy Incident Integration alert actions in the Splunk Add-on for BMC Remedy Using SOAP API | Lookups for the Splunk Add-on for BMC Remedy |
This documentation applies to the following versions of Splunk® Supported Add-ons: released
Feedback submitted, thanks!