Best practices for integrating content with Splunk Security Essentials

The following methods can help you seamlessly integrate custom content as a third-party developer.

Add a configuration to avoid a Splunk Enterprise restart after installation

During development, you might need to restart Splunk Enterprise to make the system recognize changes made to the configuration. However, users who install your app don't need to restart Splunk Enterprise. To avoid Splunk Enterprise restarting on users, add this configuration to the app.conf file in the default folder of your app:

[triggers]
reload.essentials_updates = simple

Ship the content through your app

If you want to ship the content through your app rather than provide automatic updates, use the following method. You don't need a web server and instead, you can ship a JSON file through your app using the appserver/static folder.

For this method, follow these steps:

1. Create an appserver/static folder, if one doesn't exist already, and put the JSON file in there.
2. In the default/essentials_updates.conf file, start the line with "SPLUNKD" so that SSE searches for local updates.
3. Apply the default.meta and app.conf settings.

This is an example of what the configuration in the default/essentials_updates.conf file looks like:

[ButtercupOfflineContent]
channel=ButtercupOfflineContent
order=10
name=Buttercup Offline Content
type=app
app_context=Splunk_Security_Essentials
content_download_url=SPLUNKD/static/app/Buttercup_App/mycontent.json

You can choose any name for your app because there isn't a naming scheme. To help users find the app on Splunkbase, include "Security Essentials" in the title.

Verify your configuration

View your content by installing the app on your local machine. After you install it, run the following search to see important details about your content:

| sseanalytics include_json=true | search channel=ButtercupLabs | table id summaries *

Populate the data inventory

One of the main configuration interfaces in SSE is the Data Inventory configuration. By default, custom content doesn't appear there. In the default case, map your data_source_categories field to standard Data Source Categories (DSCs), for example, DS0001MAIL-ET02Receive.

To have your custom content appear as a vendor-specific data-source category, include this code at the end of your configuration:

{
    "create_data_inventory": true
}

If that code is present, SSE creates a custom DSC with the data you've provided in the company_* fields.

When pullJSON.py pulls in the data_inventory.json, the python script inspects the custom_content kvstore collection, which is on the back end of the partner framework. That script includes any detections with a create_data_inventory=true configuration and processes the first piece of content found using this logic:

dscid = "VendorSpecific-" + row['channel']
baseSearch = "index=NOTAPPLICABLE TERM(No baseSearch Provided)"
legacyName = "Unknown Channel: " + row["channel"]
shortUnifiedName = "Unknown Channel: " + row["channel"]
description = "No Description Provided"
commonProductNames = []
if "company_description" in customJSON:
    description = customJSON['company_description']
if "company_name" in customJSON:
    legacyName = customJSON['company_name']
    shortUnifiedName = customJSON['company_name']
    commonProductNames.append(customJSON['company_name'])
if "company_base_spl" in customJSON:
    baseSearch = customJSON['company_base_spl']

data_inventory["VendorSpecific"]["eventtypes"][dscid] = {
    "baseSearch": baseSearch,
    "legacy_name": legacyName,
    "short_unified_name": shortUnifiedName,
    "description": description,
    "name": legacyName,
    "common_product_names": commonProductNames
}

As you work through the steps in the process of creating and bundling content into SSE, be aware of some limitations:

• Because this process only ships JSON files into SSE, you must host any logo online.
• Avoid excessive text in your content. This process requires SSE to download the content in SSE every time it refreshes, so large amounts of content can slow SSE, even though the Splunk platform compresses the content by default.
• In this SSE interface, adding demo data or different versions of searches is unsupported.

Secure app and clean data

Splunk Enterprise cleans all data provided through the partner framework to ensure the safety of users' machines. The cleaning configuration is hard coded into generateShowcaseinfo.py, the primary REST endpoint that powers SSE.

You can pull content only from the following fields in the partner content framework using the corresponding data-cleansing methods.

Field	Data type
bookmark_notes	String
bookmark_status	String
bookmark_status_display	String
bookmark_user	String
datasource	String
create_data_inventory	Boolean
datasources	String
name	String
inSplunk	String
journey	String
usecase	String
highlight	String
alertvolume	String
severity	String
category	String
description	String
domain	String
gdpr	String
gdprtext	String
hasSearch	String
mitre	String
released	String
killchain	SPL
searchkeywords	String
advancedtags	String
advancedtags	String
printable_image	String
icon	String
company_logo	String
company_logo_width	String
company_logo_height	String
company_name	String
company_description	String
company_link	String
dashboard	String
relevance	String
help	String
howToImplement	String
knownFP	String
operationalize	String
search	SPL
data_source_categories	String
mitre_technique	String
mitre_tactic	String
open_search_panel	Boolean
additional_context	Array
additional_context.title	String
additional_context.search_label	String
additional_context.detail	String
additional_context.link	String
additional_context.search_lang	String
additional_context.search	SPL
additional_context.open_panel	Boolean

Depending on the data type you work with, SSE uses one of the following cleaning methods:

• String
• Number
• Boolean
• Array
• Search Processing Language (SPL)

String

BeautifulSoup (bs4) in Python strips HTML:

obj[field] = BeautifulSoup(obj[field], "lxml").text

Number

BeautifulSoup (bs4) in Python strips HTML:

obj[field] = BeautifulSoup(obj[field], "lxml").text

Boolean

Use this statement to check for a Boolean data type and delete the statement if it doesn't use Boolean:

if not isinstance(obj[field], bool):
    debug.append({"status": "WARN", "msg": "clean_content, deleting field because it's not actually a bool", "path": path, "field": field, "value": obj[field]})
    del obj[field]

Array

Make sure each row in the array is an object. Use this statement to recursively call the clean_content function.

if key_checking[path + field] == "array":
    for i in list(range(0, len(obj[field]) )):
        if isinstance(obj[field][i], object):
            obj[field][i] = clean_content(obj[field][i], key_checking, path="")

SPL

Pass input into SSE through jQuery and into a pre tag using .text(), which ensures that no code runs.

Troubleshoot data-cleaning issues

In any case where you need to troubleshoot the ShowcaseInfo.json file, check the debug output for logs using the JavaScript console. Any time a field is deleted from SSE, it logs the event in the debug output:

require(['json!' + $C['SPLUNKD_PATH'] + '/services/SSEShowcaseInfo?bust=' + Math.random()], function(showcase){
    window.debug_showcase = showcase; 
    console.log("Got Showcase", showcase.debug); 
})