Create and manage database outputs

A database output object lets you define how to send data from Splunk Enterprise to a database on a recurring basis. Defining database outputs is useful if you want to store your indexed Splunk Enterprise data in a relational database.

Create a database output

1. From within Splunk DB Connect, click the Datalab > Outputs tab.
2. Click New Output.
3. Name output: Name and describe your new output.
4. Search for Splunk fields to output: Perform a Splunk Enterprise search to select the fields to output to your database.
5. Map Splunk fields to table columns: Map the Splunk Enterprise fields you selected to columns in your database.
6. Preview your output: Preview the output by verifying it in a table.
7. Finalize your output: Specify how often DB Connect sends the data, then save the output.

Name output

• Name: The output name cannot contain any spaces. Do not use special characters in the output name.
• Description
• App: The name of the Splunk Enterprise app in which DB Connect saves this output object. By default, the pop-up menu selects Splunk DB Connect. This menu enables other apps to use DB Connect inputs, outputs, and lookups within their own context.
• Connection: Choose the database connection you want to use with this output. DB Connect validates the connection and displays an error message if it is not able to do so. You cannot continue the new output setup process unless you have specified at least one valid connection the Connection dialog.

Search for the Splunk Enterprise fields to output

In the second step, you define the Splunk Enterprise fields that you want to output to the database table as columns:

1. Perform a search of your Splunk Enterprise data. You can either enter a search using the Search Processing Language (SPL), or you can run a report (saved search).
2. To select a saved search, click Saved Search, then click the field that appears. Then, either enter the name of the saved search to use or choose it from the menu. Splunk Enterprise performs the search and displays the results in a table. Each column corresponds to a Splunk Enterprise field.

   Notes: You can fine-tune the format of your output directly from this search field by using standard search commands. For example, if you want Splunk Enterprise to display and use output to send data to your database as key-value pairs, use the eval search command here. If you want to change the quoting pattern, use rex.

   If you want Splunk Enterprise to send a specific number of rows each time the output runs, use the head search command. For example, to specify that the output should send no more than 1000 rows each time it runs, define your search as follows:

   index=main sourcetype=foo status=ERROR | head 1000

   If you want to use the Saved Search, set the permission of the saved search to This app only (splunk_app_db_connect) or All apps.

3. From the results that appear, click all the columns that correspond to the Splunk Enterprise fields that you want to include in your database table as a column. The blue Fields Selected counter keeps track of how many fields you have chosen.
4. When you have chosen all the fields you want, click Continue.

Map Splunk Enterprise fields to table columns

In the third step, you map the Splunk Enterprise fields you chose in the previous step to columns in your database table.

1. From the corresponding dialog menus, choose the catalog, schema, and table that contain the columns you want to map the fields to.
2. From each pop-up menu under the Fields heading, choose the Splunk Enterprise field that you want to map to the corresponding column under the Columns heading. You can choose Skip this Column to ignore the corresponding column when mapping data.
3. Click Continue.

Preview your output

In the fourth step, DB Connect offers you a preview of the final output in a table. The table displays the database columns you mapped. Splunk Enterprise data populates the table.

1. Click on a previous step to change fields or mappings.
2. When you are satisfied with the output, click Continue.

Finalize your output

In the last step, finalize the output.

1. In the Execution Frequency field, enter the number of seconds between output executions. Alternatively, you can enter a valid cron expression. For example, enter 120 to instruct DB Connect to wait two minutes after it has sent data to your database before doing it again. Be aware that DB Connect only sends new events at each execution.
2. Click Save.

Edit database outputs

To see a list of the database outputs you defined, navigate to the Configuration > Outputs tab.

To edit a database output, click its name. You can make changes to a database output using the following buttons on the output page:

• Enable/Disable: Enable or disable an output.
• Edit: Edit an output by clicking its name or the Edit button.
• Clone: Create a copy of the output. You must give the copy a unique name.
• Delete: Delete the output.

You can also edit any of the attributes of a database output listed in Create a database output, except its name. To change the name of an output, clone it, give the clone the name you want, and then delete the original output.

Enable output to multi-byte character sets

DB Connect can send data that is in a multi-byte character set, such as Traditional Chinese, using a database output. Depending on your database, you may need to change certain settings to the database to properly receive and store the data.

• MySQL: When creating a connection to a MySQL database, customize the JDBC URL by adding some additional query parameters. For more information, access MySQL documentation for Inserting unicode UTF-8 characters into MySQL.
• PostgreSQL: By default, this database supports multi-byte character sets. You do not need to take additional steps.
• Microsoft SQL Server: Ensure your database columns' data types are N-variant types, such as NVARCHAR versus VARCHAR).
• Oracle: Change your database character set to AL32UTF8. For more information, access Oracle's documentation for Supporting Multilingual Databases with Unicode.
• Other databases: Consult your database's documentation for more information about enabling multi-byte character sets.

Use database outputs

Database outputs run automatically at the frequency you set during the "Finalize your output" step of the setup process. To verify that database outputs are working properly, query your database after a few executions of the output operation to ensure that DB Connect is sending your Splunk Enterprise data properly.

DB Connect 3 does not support running scheduled task (input or output) on the search head in the Search head cluster deployment. You must run the scheduled task on a heavy forwarder.

Use modular alert to run database output

DB Connect provides a modular alert which allows users to actively respond to events and send alerts. You can configure the DBX output alert action on the Alert Actions Manager page. See alert action manager.

To use the DB Connect modular alert:

1. Navigate to the Search page in DB Connect.
2. Create a search, then select Save as>Alert.
3. Enter the title and optional description.
4. In the Trigger Actions field, select DBX output alert action.
5. Enter the Output Name. The output name must exist in DB Connect.

For details about custom alert, see Create custom alert.

Use dbxoutput command to run database output

dbxoutput is a search command you can use to run database outputs that you have defined in DB Connect.

Syntax

   dbxoutput output=<string>

Required Arguments

output

Syntax: output=<string>

Description: Name of a configured database output object.

Example

The following example uses the output dbx_output to send the results of a search query to a database.

<search query> | dbxoutput output="dbx_output"