Splunk® DB Connect

Deploy and Use Splunk DB Connect

Download manual as PDF

This documentation does not apply to the most recent version of DBX. Click here for the latest version.
Download topic as PDF

What's new in Splunk DB Connect

New Features

Performance improvement

Under similar hardware conditions and environment, DB Connect 3 is about 2-10 times faster than DB Connect 2 depending on the task you perform. See performance expectations for more detailed information on performance improvement.

Usability improvement

DB Connect 3 improves usability with the following features:

Health dashboard improvement

New DB Health and Input Metrics dashboards allow you to monitor and troubleshoot several aspects of your database connections from inside Splunk Enterprise. See Monitor database connection health.

Feature improvement

DB Connect 3 supports SQL stored procedures in the dbxquery. See dbxquery on how to use the dbxquery command.

What's changed in DB Connect 3.0.0

Separated configuration files

Instead of configuring inputs, outputs and lookups in one inputs.conf file, in DB Connect 3, inputs, outputs and lookups are configured in discrete files db_inputs.conf, db_outputs.conf and db_lookups.conf. This increases configurability and performance. Note that the rising column checkpoint is no longer stored in the inputs.conf configuration file. See Configuration file reference for detailed description and example of each .conf file.
If you are running DB Connect version prior to DB Connect 3, see Migrate DB Connect to DB Connect 3 for detailed migration procedures.

Retry policy on scheduled task failures is changed

Rapid retry policy of scheduled tasks for failing inputs and outputs is disabled. DB Connect 3 does not retry failed tasks until the next interval. Consequently, the auto_disable setting is deprecated; it will not cause configuration validation errors, but it is no longer functional.

Resource pooling is removed

The resource pooling feature of DB Connect 2.0 through 2.4 was designed to increase the speed and efficiency of database input and output operations by distributing jobs horizontally from the master node to one or more resource pool nodes. With the increased vertical scale of DB Connect 3, horizontal resource pooling is not required and has been removed.

Scheduled tasks are disabled on search head cluster

DB Connect 3 does not support scheduled inputs and outputs in a search head cluster deployment since it may cause some duplicates or data missing. Splunk recommends that you run scheduled inputs and outputs from a heavy forwarder. Error messages will appear when attempting to schedule DB Connect inputs or outputs on a search head cluster member.

  • You can still run ad hoc DB Connect commands, such as dbxquery, dbxlookup, or dbxoutput on a search head cluster using the search bar to create dashboards or modular alerts.
  • If you have scheduled tasks (inputs and outputs) in a search head cluster on DB Connect 2, you must back up the configuration files on a heavy forwarder when migrating to DB Connect 3. See Migration for more details.
  • If you have lookups running in a search head cluster on DB Connect 2, keep them running on search head cluster in DB Connect 3.

Redesigned input types

The advanced input type is merged into rising column input type. Advanced input type is not available in DB Connect 3. See Choosing an input type for details. Note that rising column checkpoints are stored in splunk/var/lib/splunk/modinputs/server/splunk_app_db_connect, each input has a separate checkpoint file.

.conf files

Some fields have been removed or added in the .conf files. See Configuration file reference.

Directory change for JDBC JAR files

DB Connect 3 changes the layout of Java JAR files to separate internal files from drivers which administrators need to work with. To install a JDBC driver, extract the JAR file to $SPLUNK_HOME/etc/apps/splunk_app_db_connect/drivers/. When adding complex drivers with multiple JAR file requirements, the main JDBC JAR file and the extended JAR files must be installed in different folders. The path for the extended JAR files is $SPLUNK_HOME/etc/apps/splunk_app_db_connect/drivers/<JDBC driver name>-libs. For upgraders, DB Connect 3 setup and migration will move simple JAR files correctly. If you are using Spark SQL, or a Teradata or Oracle database, you must move dependent JAR files manually. See Install database drivers for more detail.

dbxlookup command

Automatic and scripted lookups are replaced by a simpler, more performant dbxlookup command. You cannot perform scripted or automatic lookups using the lookup command any longer in DB Connect 3, unless this is to access a standard Splunk lookup populated via dbxquery. To access database tables for lookup purposes directly from a Splunk search, use the dbxlookup command. See Create and manage database lookups for more detail.

dbxquery command

The following options were changed in the dbxquery command:

  • The wrap argument is now deprecated and will have no effect. The value for wrap is now set to False by default. Query wrapping as introduced in version 2.1 is no longer an option, but can still be introduced through manual query editing in Editor mode.
  • The output argument is now deprecated and will have no effect. The value for output is now set to CSV by default. JSON output is no longer available.
  • The shortnames option now defaults to True.
  • When a table name is not available for a query, driver, or database, "NA" is used as the table name. Previously, a NULL character was used.

dbxoutput command

In versions 2.3.0 to 2.4.0, if the related database output is disabled, the dbxoutput command is also disabled. In DB Connect 3, the dbxoutput command works regardless of output status.

UI Changes

The main updates for UI in DB Connect 3 include:

  • Reorganize the main tasks of DB Connect Inputs, Outputs and Lookups under Data Lab.
  • Group general settings and database settings under Configuration.
  • Provide SQL Explorer to query and explore data.
  • Provide graphical logging level configurations.

DB Connect 3

  • The Data Lab page

Dbx30newdatalab.jpg

  • The Configuration page

Dbx30newui.jpg

  • The SQL Explorer page

Dbx30sqlexplore.jpg

Known issues and fixed issues

For information about bug fixes and know issues, see the Release Notes.

PREVIOUS
About Splunk DB Connect
  NEXT
Share data in Splunk DB Connect

This documentation applies to the following versions of Splunk® DB Connect: 3.0.0, 3.0.1, 3.0.2, 3.0.3


Comments

Hi imdiconectiv
Sorry for the confusion.
For the question you asked in your comments, You can run dbxquery in a saved search run on search head cluster for alerts or reports. What you cannot do is to ingest data read from a DB on a search head cluster.
Similar answer has been posted at: https://answers.splunk.com/answers/519944/scheduled-inputsoutputslookups-on-db-connect-v302.html
I have updated the document to make it more clear to the user.
Thanks for your feedback.

Rwang splunk, Splunker
May 25, 2017

This also makes no sense to me. When the manual says "DB Connect 3 does not support scheduled inputs and outputs in a search head cluster deployment. Splunk recommends that you run scheduled inputs and outputs from a heavy forwarder." .

Does that mean that you cannot run reports using dbxquery from a search head cluster on a scheduled basis?

I can understand that you might not be able to run a cron there that pushes data into splunk, but just running a query that connect the db make little sense.

Neither does moving that function to a Heavy Forwarder. Heavy forwarder typically send data to indexers, and are not used for reporting.

The manual needs to be clearer on this.
I added this same under "was it useful" and under another user quesiont on answers:
https://answers.splunk.com/answers/519944/scheduled-inputsoutputslookups-on-db-connect-v302.html

Jimdiconectiv
May 25, 2017

http://docs.splunk.com/Documentation/DBX/3.0.3/DeployDBX/WhatsnewinSplunkDBConnect#Scheduled_tasks_are_disabled_on_search_head_cluster and under was it useful.

Jimdiconectiv
May 25, 2017

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters