Collect observables using Splunk Intelligence Management

Splunk Intelligence Management identifies and collects observables from data sources such as emails, spreadsheets, reports, and other submissions. Collecting observables is a complex and ongoing process since data has many formats and structures. An example of structured data is JSON and an example of unstructured data is emails.

For more information on supported observables, see Observables supported by Splunk Intelligence Management.

Identify data extraction issues

Use the following table to identify issues with data extraction:

Issue	Details
URL extraction issue	URLs might not parse correctly for the following reasons: When the URL contains parentheses such as () When the URL contains bracketed colons such as "[:]" For example: yahoo.c instead of yahoo.com is an example of incomplete extraction. A fully qualified domain name might be classified as a URL.
Filename extraction issue	Filename might not parse correctly when it contains spaces.
Disambiguation between scripts and domains	Domain is incorrectly categorized as a Perl script. For example: myacmecompany.pl Domain is incorrectly classified as a Python script. For example: myacmecompany.com.py

Splunk Intelligence Management validates IPv4 addresses. However, if the IP addresses are in the range of private IP addresses, the IP is not extracted as an indicator of compromise (IOC).

The following types of IP addresses are not extracted:

• loopback address such as 127.x.x.x
• site local address such as 10/8 prefix, 172.16/12 prefix, 192.168/16 prefix
• value of 0

Improve data collection using Splunk Intelligence Management

Splunk Intelligence Management uses Apache Spark when handling massive amounts of data extraction and normalization while monitoring data quality and performance.

Splunk Intelligence Management improves data extraction through the following key initiatives:

• Allow users to submit structured indicator objects
• Process and prioritize email submissions by upgrading the extraction engine to use Apache Spark
• Improve the URL data model. Improving the data model includes splitting URLs into different components, capturing them correctly, and separating domain name concepts from URLs.

Observables supported by Splunk Intelligence Management

Splunk Intelligence Management identifies the following observables:

Entity	Type
REGISTRY KEY	Observable
IPV6	Observable
IPV4	Observable
CIDR BLOCK	Observable
URL	Observable
MD5	Observable
SHA1	Observable
SHA256	Observable
BITCOIN ADDRESSES	Observable
SOFTWARE	Observable
EMAIL ADDRESS	Observable
PHONE NUMBERS	Observable
DOMAIN	Observable
CVE (based on NIST's CVE standard)	Attribute
MALWARE	Attribute
THREAT ACTOR	Attribute
MITRE ATT&CK	Attribute

Your account owner must enable phone numbers based on enclaves because they are not extracted by default.