Splunk® Supported Add-ons

Splunk Add-on for Check Point OPSEC LEA

Download manual as PDF

Download topic as PDF

Troubleshoot the Splunk Add-on for Check Point OPSEC LEA

General troubleshooting

For helpful troubleshooting tips that you can apply to all add-ons, see Troubleshoot add-ons in Splunk Add-ons. For additional resources, see Support and resource links for add-ons in Splunk Add-ons.

Splunk Add-on for Check Point OPSEC LEA logs

This add-on has 3 logs that are located at $SPLUNK_HOME/var/log/splunk:

  • splunk_ta_checkpoint-opseclea_modinput.log
  • splunk_ta_checkpoint-opseclea_ucc_lib.log
  • splunk_ta_checkpoint-opseclea_util.log

To check for errors in the internal logs for this add-on, you can perform this search:

index=_internal source=*ta_checkpoint-opseclea*

You can configure the logging verbosity on the configuration page for the add-on. Supported log levels are INFO, DEBUG, and ERROR.

A large amount of debug information is provided by the Check Point OPSEC SDK if the log level is set to DEBUG. This will have an impact on performance, especially if the data volume is large.

Online mode inputs limitation

Inputs are executed in a thread pool which can handle between 4 and 128 tasks. However, because online mode tasks never finish, each online mode task always occupies one thread in the thread pool. The add-on can support 120 online mode tasks at the most. If there are more than 120 online mode tasks, the data collection process will exit and the following error message will be logged: "There are too many online mode inputs. The count is <int> which should be less than 1".

Use this search to check if you are encountering this limitation.

index="_internal" sourcetype="opseclea:log:modinput" "too many online mode inputs"

To resolve this error, deploy rest inputs on another heavy forwarder.

32-bit run time environment errors

This add-on requires a 32-bit run time environment. If the latest i686 packages are not the same version as the currently installed x86_64 packages on the machine, errors like the following can occur:

Error: Protected multilib versions: libgcc-4.4.7-3.el6.i686 != libgcc-4.4.6-4.el6.x86_64
Error: Protected multilib versions: pam-1.1.1-13.el6.i686 != pam-1.1.1-10.el6_2.1.x86_64

To resolve this type of error, you need to upgrade the x86_64 packages first and then install the i686 packages.

For example:

> yum upgrade libgcc.x86_64 pam.x86_64

> yum install pam.i686 glibc.i686 libgcc.i686

SIC errors

You can use the following search to determine if SIC errors are occurring.

index="_internal" sourcetype="opseclea:log:modinput" "SIC ERROR"

Follow these steps to troubleshoot SIC errors:

  1. Check that the Log Server Type (lea_server_type) is configured correctly. The valid options are primary, secondary, and dedicated. Different log server types have different entity sic names.
  2. Check that the opsec_entity_sic_name and opsec_sic_name are correct.
    The opsec_sic_name uses this format: CN=<Splunk OPSEC application name>,O=<host information>..<a string>.
    For opsec_entity_sic_name, use the following guidelines:
    The Entity SIC Name of the Log Server object can be created from the CN in the SIC Name.
    • If the Log Server is activated on the Primary Management Server, replace it with "cp_mgmt" as in this example:
    CN=cp_mgmt,O=CP-hero1-take-195.checkpoint.com.fsd9kx
    • If the Log Server is activated on the Secondary Management Server, replace it with "cp_mgmt_<OBJECT_NAME>" as in this example:
    CN=cp_mgmt_<OBJECT_NAME>,O=CP-hero1-take-195.checkpoint.com.fsd9kx

    where <OBJECT_NAME> is the name of the Log Server object.
    • If the Log Server is activated on dedicated server, replace it with "OBJECT_NAME" as in this example:
    CN=<OBJECT_NAME>,O=CP-hero1-take-195.checkpoint.com.fsd9kx

    where <OBJECT_NAME> is the name of the Log Server object.
  3. If you receive the following error message in the logs: "Invalid arg: file %s doesn't exist", check that a file with the extension .p12 exists in the $SPLUNK_HOME/etc/apps/Splunk_TA_checkpoint-opseclea/certs directory. If not, pull a cert down by running pull-cert.sh or delete the connection and add a new one on the add-on configuration page.
  4. If other SIC errors are present in the log, see the Check Point documentation at http://dl3.checkpoint.com/paid/20/How-To-Troubleshoot-SIC-related-Issues.pdf?HashKey=1463490738_979d1a6f694300a70576eecfe9d55b85&xtn=.pdf.

lea_loggrabber errors

If you are seeing lea_loggrabber errors such as "COMM_FAILURE" in the add-on logs, check the following:

  • The machine running this add-on can access the Check Point machine.
  • The Check Point machine is running and healthy.

Time configuration

If you configured your OPSEC LEA connections and inputs in the configuration files and you receive the error "Required syntax: 'starttime=YYYY-MM-DDThh:mm:ssTZD", you need to configure the time using ISO-8601 format.

Historic log fetching limitation

There are limits to the add-on's ability to collect historic log data when more than one log file exists on Check Point. When using the "Start Time" parameter, data collection will start at the specified time from the currently active log file. The start time does not apply to other log files.

Events lost when forwarder stopped

If a forwarder is stopped during data collection from OPSEC LEA, events can be lost. If you need to stop a forwarder for any reason, disable the Splunk Add-on for Check Point OPSEC LEA first, and wait 10 seconds before stopping the forwarder.

Log Server stops forwarding logs to LEA clients in Check Point version R77.10

In Check Point version R77.10, the Log Server may stop forwarding logs to LEA clients. This occurs when switching the active log to a new one (log switch) and the Log Server does not notify the LEA client about the new log file.

Fix: Install hotfix on Check Point Multi-Domain Security Management Server/Log Server machine.

Integer overflow for bytes values

Due to an OPSEC SDK limitation, some bytes values are beyond the range of the integer type. To work around this issue, bytes related values that have a negative value as a result of EVAL (e.g., bytes, send_bytes, client_inbound_bytes, client_outbound_bytes, server_inbound_bytes and server_outbound_bytes) will be converted to INT_MAX (2147483647) at search time. The raw data will contain the negative value, but in the extracted field, the negative value will be converted to 2147483647.

Missing OPSEC fields

While configuring or modifying input, if some fields are added to Excluded Fields and the Fetch all fields option is selected, the fields added to Excluded Fields are filtered out. To ensure all OPSEC fields are indexed, select Fetch all fields and make sure that no fields are added to the Excluded Fields section.

Certificate authority not found error

If you are seeing this error Opsec error. rc=-1 err=-93 The referred entity does not exist in the Certificate Authority, reset the one-time password from the OPSEC configuration to pull the certificate again. This is commonly a result of using a hyphen in the OPSEC app name.

Extra fields in your Splunk events

If you are upgrading the Splunk Add-on for Check Point OPSEC LEA from 4.2.0 to 4.3.0, you may see more fields in your Splunk events that are not present in the Selected Field lists. Manually add those fields into the Excluded Fields box to exclude them.

Upgrade instructions using the command line and configuration files.

Checkpoint errors

When two or more data inputs are configured for the same product (e.g. non_audit, firewall) and follow similar naming convention (e.g input/9, input/19, etc), there is a possibility of race condition as all these inputs refer to the same checkpoint file.

You will see the following warning message: Unable to get a lock or parse the checkpoint file <Name of the checkpoint file>. Will retry on next run.

Your error will be logged in splunk_ta_checkpoint-opseclea_modinput.log and Splunk software will stop indexing data for that particular data input.

Data collection errors persist after upgrading to version 4.3.1

If data collection errors persist after upgrading the Splunk Add-on for Check Point OPSEC LEA from 4.3.0 to 4.3.1, create a new data input with a new OPSEC application object and enter the last event index time in the Start time input field.

  1. Disable the inputs.
  2. If the Data input field is set to audit, enter the following search query and make a note of the last event index time:

    index=<index_name> sourcetype=opsec:audit | head 1 | eval latest_timestamp=strftime(time, "%Y-%m-%dT%H:%M:%STZD") | table latest_timestamp

    Replace TZD with the local timezone.

  3. If the Data input field is set to non-audit, enter the following search query and make a note of the last event index time:

    index=<index_name> sourcetype!=opsec:audit | head 1 | eval latest_timestamp=strftime(time, "%Y-%m-%dT%H:%M:%STZD") | table latest_timestamp

    Replace TZD with the local timezone.

  4. Upgrade the Splunk Add-on for Check Point OPSEC LEA.
  5. Create a new data input with a new OPSEC application object, and enter the last event index time value in the Start Time input field.
  6. Restart Splunk.
Last modified on 22 October, 2019
PREVIOUS
Configure the Splunk Add-on for Check Point OPSEC LEA using the command line and configuration files
  NEXT
Lookups for the Splunk Add-on for Check Point OPSEC LEA

This documentation applies to the following versions of Splunk® Supported Add-ons: released


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters