Overview of Splunk

Overview of Splunk

Splunk is search software for any type of data. Learn more about how Splunk works by reading through this intro page. You'll find many links here for installing, configuring and customizing your Splunk installation.

Configuration options

Most of Splunk's configurations can be reached through Splunk Web. You can use the CLI as well. Splunk also comes with configuration files for advanced customization.

Note: Many of Splunk's settings can be configured in multiple ways. Decide which works best for your setup. Some advanced features may only be configured through configuration files.

Installation and upgrade

Installing Splunk is easy and fast. Here are instructions for installing, upgrading, or backing up an existing copy.

• Installation
  • Installation instructions for all supported platforms can be found in the Installation Manual
  • On *nix platforms, use a tarball or RPM file.
  • On Mac, use a tarball or DMG file.
  • On Windows, download the .exe file and install. Instructions for Windows install are located here.
• Upgrade
  • There are a lot of new features available in 3.2. You may want to consider an upgrade if you are running an earlier version.
  • Upgrade instructions are here.
• It's a good idea to back up your current instance before you upgrade.

Data inputs

Splunk can receive data in a variety of ways. Each configuration change can be affected via:

• Splunk Web
  • Upon first run, Splunk contains a link to enable tailing /var/log.
  • Configure other inputs through the Admin section of Splunk Web.
• CLI
  • Use Splunk's CLI to set up data inputs.
• Configuration files
  • Use inputs.conf to configure advanced input settings.

Read on for a brief description of each input type.

• Tail
  • Use tail to stream live data into Splunk.
  • Tail works for continuous inputs from files or directories.
• Upload
  • Upload a file directly to Splunk Web.
  • Files can be local to the workstation or on the Splunk server.
  • Note: If you want to upload a file directly to Splunk Web, click the upload button in the Inputs section of Splunk Web's Admin interface.
• Network ports
  • Splunk supports UDP and TCP connections.
  • Configure syslog on UDP 514.
  • Use TCP connections for log4j.
• Distributed
  • One Splunk Server can receive data from any number of other Splunk Servers via data distribution (description below).
  • This port is configurable, but defaults to 9998.
• Scripted inputs
  • Use scripted inputs to receive the outputs of command-line tools (such as vmstat, iostat, netstat, top, etc.) or other programs.
  • Learn more about scripted inputs.

Note: For a more in-depth description of inputs, read how input configuration works.

Distributed data

Configure distributed inputs and outputs across your network. Send data between one Splunk instance and another, or third party software. For an overview on all the available configuration options, see How data distribution works.

• Forwarding and receiving
  • A Splunk Server in forwarding mode can send data to one or more Splunk instances.
  • Any Splunk Server can receive data from one or more Splunk instances.
  • Learn more about forwarding and receiving.
• 3rd party systems
  • Splunk can also forward raw data to any other system or software.
  • You can set up Splunk to send or receive data from 3rd party systems. Learn how.

Indexing

Splunk takes all data from inputs and sends it to an indexing pipeline. Data is then broken up into separate events via segmentation rules. Most data is segmented and timestamped correctly. However, you may wish to configure Splunk to index your data in particular ways. Learn more about how indexing works.

Here are some things you might want to consider:

• Configure event boundaries
• Configure segmentation
• Mask sensitive data
• Character set
• Timestamp recognition

Configuration for indexing is set mostly through props.conf and transforms.conf

Fields

Fields are a useful aspect of Splunk's search interface. You can use Splunk's built-in fields that are enabled by default. Here's a list of Splunk's default fields, including links to more in-depth documentation:

• Host:
  • Host is the label for the device that originated the event.
  • Read more about host.
• Source type
  • A source type refers to any common format of data produced by a group of sources, such as weblogic or syslog.
  • Learn more about source types.
• Event types
  • Event types are groups of common events.
  • Learn more about event types.

You can also create your own fields. Custom fields are useful for:

• customizing searches (see below for search options).
• creating field actions.
• enabling event type templates.

To learn more about creating custom fields, see how fields work.

Search

Splunk's search interface is useful for tracking down different aspects of your data. Here are a few things you can do with your searches:

• Search commands.
  • Splunk has a powerful search language.
  • Craft simple to sophisticated searches.
• Save searches.
  • Any search can be saved and run at any time.
  • Save searches with variables to fill in at search time, including:
  • Form search.
  • Macro search.
• LiveTail
  • Run a search to watch data in real time as it's indexed, like tail -f in *nix.
  • Read more about Live Tail.
• Alerts
  • Schedule Splunk to send search results via email or RSS.
• Transactions
  • Search for transactions that occur across events, such as email threads, store purchases

For a more detailed overview of search, see how search works.

Distributed search

In a distributed set up, you may want to search across multiple instances of Splunk. Enable distributed search to federate searches across your entire Splunk deployment. Read more about how distributed search works.

Security

Secure your Splunk Server with the following security configuration options. Here's a brief overview of the available features. For a more detailed overview, see security options.

Authentication

Splunk includes several authentication options, including:

• Roles, which allow you to set up:
  • User roles capabilities.
  • User-based access controls.
• LDAP
  • Set up LDAP.
• SSL
  • Enable HTTPS.
  • Or SSL for Splunk's back-end.

Audit

Splunk 3.2 includes new audit features. Use the following options to enable separate auditing configurations:

• File system monitor
  • The file system change monitor watches any designated file system and sends an event if files or directories are affected in any way.
  • By default, Splunk monitors its own $SPLUNK_HOME/etc/ directory for configuration changes.
• Audit events
  • Events generated by the file system change monitor as well as user activity within Splunk.
  • Audit events are stored in a separate index, _audit.
• Audit event signing
  • Set up cryptographic signing for audit events.
• IT data signing
  • Enable cryptographic signing for all your events as they enter Splunk.
• Archive signing
  • Sign your data as it is archived.
• Event decorations
  • Mark your audit events with icons so they're more noticeable.

Data management

Splunk Servers often index large amounts of data each day. You may want to enable advanced settings to handle the following data management scenarios.

• Index management, including:
  • Add or Remove an index.
  • Delete data from the index.
  • Move an index.
• Data archiving, including:
  • Set retirement policy.
  • Automate archiving.
  • Restore archived data.
  • Export event data.
• Storage options:
  • Disk usage.
  • Use separate partitions for Splunk's data store.
  • Use WORM (Write Once Read Many) volumes for Splunk's data store.

NOTE: Many data management settings are enabled on a per-index basis, using indexes.conf. To learn more about indexes, see how indexes work.

Deployment server

In a distributed set up, enable one or more Splunk instances as deployment servers. A deployment server pushes out configuration changes to other Splunk instances.

For a complete overview of all deployment options, read the Deployment manual. For instructions on configuring and enabling the deployment server and clients, read the Admin manual section on the deployment server.

Performance tuning

The following options help you tune Splunk's performance for your environment. Depending on your system and requirements, you may want to change one or more of the following settings:

• Indexing
  • Change various configurations to speed up Splunk's intake of data.
• Search
  • Settings for faster return of search results.
• Storage efficiency
  • Cut down on the space of your Splunk index.
• CPU and memory footprint
  • Tune Splunk's CPU usage and memory settings.
• Backup
  • Back up your Splunk install.
  • NOTE: It is a good idea to backup Splunk before performing any migrations or upgrades.

A more in-depth overview of performance tuning options is available here.

Configuration files

Many of Splunk's advanced configurations and customizations are available only through configuration files. Create configurations by copying files into a custom bundle directory. Learn more about bundle directories and configuring bundle directories.

Applications

Applications are directories of configuration files with specific purposes. Configure your own applications by following these instructions.

You can also share your configuration file directories as applications with the Splunk community on SplunkBase.

Customization

Pimp your Splunk! Everybody's data is a little bit different. Maybe you want to set custom configurations for the system you're running Splunk on. Here are options for personalizing your Splunk instance.

Splunk Web appearance

Change various aspects of Splunk Web's appearance:

• Dashboards
  • Configure user settings and dashboards via prefs.conf.
• Decorations
  • Set icons for event types with dynamic event rendering.
• Literals
  • Change the externalized strings in Splunk Web via literals.conf
• Skinning
  • Change the way your web interface looks.
  • Read the Developer's Guide for help with skinning Splunk.

Extend Splunk

Splunk includes a REST API. Read the Developer's Guide to learn more about the REST API. To configure additional REST endpoints, use restmap.conf.

Troubleshooting

If there's something you need help with, even after reading the documentation, contact Splunk support.

If there's a feature you don't see here that you want included, file an enhancement request with Splunk support.

We're always interested in your feedback.

• Splunk support.
• Splunk forums.

• Was this topic useful?
• Post a comment