Splunk® Enterprise

Installation Manual

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

About upgrading to 6.4 READ THIS FIRST

This topic contains important information and tips about upgrading to version 6.4 from an earlier version. Read it before attempting to upgrade your Splunk Enterprise environment.

Splunk App and Add-on Compatibility

Not all Splunk apps and add-ons are compatible with Splunk Enterprise 6.4. If you plan to upgrade to this release, visit Splunkbase to confirm that your apps are compatible with Splunk Enterprise 6.4.

Upgrade clustered environments

To upgrade an indexer cluster, see Upgrade an indexer cluster in the Managing Indexers and Clusters manual. The instructions in that topic supersede the upgrade material in this manual.

To upgrade a search head cluster, see Upgrade a search head cluster in the Distributed Search manual. The instructions in that topic supersede the upgrade material in this manual.

Upgrade paths

Splunk Enterprise supports the following upgrade paths to Version 6.4 of the software:

  • From version 6.0 or later to 6.4 on full Splunk Enterprise.
  • From version 5.0 or later to 6.4 on Splunk universal forwarders.

If you run a version of Splunk Enterprise prior to 6.0, upgrade to 6.0 first, then upgrade to 6.4. Users of Splunk Enterprise 5.0 also have the option of upgrading to versions 6.0, 6.1, 6.2, or 6.3 before upgrading to 6.4. See About upgrading to 6.0 - READ THIS FIRST for tips on migrating your instance to version 6.0.

Important upgrade information and changes

Here are some things that you should be aware of when installing the new version:

Distributed search now defaults to a single protocol

In an effort to reduce the potential of problems when search heads connect to search peers, several variables have been added or changed in distsearch.conf that control this process.

  • The trySSLFirst attribute no longer has any meaning in the context of search head-to-search peer connections.
  • A new attribute defaultUriScheme controls what protocol search heads use to connect to search peers, and can be set to http or https. This attribute acts as the default connection scheme for any peers that you add to a search head after you set the attribute.

After you upgrade, review distsearch.conf to confirm that the file has been updated with the new variables.

Migration time might increase significantly if there are a large number of data model or report acceleration summaries

When you upgrade to version 6.4 of Splunk Enterprise, the software generates checksums for data model and report acceleration summaries as part of the migration. This action is for better compatibility with indexes on indexer clusters, but happens on all deployments. If your deployment has a large number of existing data model or report acceleration summaries, the checksum generation process might take a long time. Splunk Enterprise generates entries in migration.log during the process:

Generating checksums for datamodel and report acceleration bucket summaries for all indexes.
If you have defined many indexes and summaries, summary checksum generation may take a long time.
Processed 1000 out of 10007 configured indexes.
Processed 2000 out of 10007 configured indexes.
[...]
Processed 10000 out of 10007 configured indexes.
Finished generating checksums for datamodel and report acceleration bucket summaries for all indexes.

The working directory for the inputcsv, outputcsv, and streamedcsv search commands has changed

The working directory for the inputcsv, outputcsv, and streamedcsv search commands has changed. When you execute these search commands after an upgrade, Splunk Enterprise stores and reads the files they create in $SPLUNK_HOME/var/run/splunk/csv, rather than $SPLUNK_HOME/var/run/splunk.

The upgrade process moves any existing working files to the new directory and logs the following message to migration.log:

Creating $SPLUNK_HOME/var/run/splunk/csv and moving inputcsv/outputcsv files into the created directory.

Note the following migration issues:

  • Apps, add-ons, or scripts that use the commands or that reference the old working directory could be negatively affected when you upgrade due to the changed directory location.
  • You must manually migrate any files that you use in conjunction with inputcsv that do not end with the .csv file extension, or that are in a subdirectory.
  • If you have a component that is external to Splunk Enterprise that uses the outputcsv command, you must manually update the paths of any files or scripts in that component that use the command.
  • Additionally, if the component contains files that outputcsv has generated, and those files either do not end in .csv or are in a subdirectory, you must migrate those files to the new working directory manually.

Search commands that exist only in a user context will no longer execute

If you have any search commands that run in the context of a specific Splunk Enterprise user (meaning that the commands have been defined in a commands.conf only for that user, for example, $SPLUNK_HOME/etc/users/alice/local/commands.conf), those commands will no longer be available for execution after you upgrade.

To fix the problem, move the command configurations to either the app level (put the configurations into $SPLUNK_HOME/etc/apps/<app_name>/local/commands.conf) or the system level ($SPLUNK_HOME/etc/system/local/commands.conf).

Confirm that the introspection directory has the correct permissions

If you run Splunk Enterprise on Linux as a non-root user, and use an RPM to upgrade, the RPM writes the $SPLUNK_HOME/var/log/introspection directory as root. This can cause errors when you attempt to start the instance later. To prevent this, chown the $SPLUNK_HOME/var/log/introspection directory to the user that Splunk Enterprise runs as after upgrading and before restarting Splunk Enterprise.

The Splunk Web visualizations editor changes take precedence over existing 'rangemap' configurations for single-value visualizations

If you use the rangemap search command to define ranges and colors for single-value visualizations on dashboards, use the Format editor instead when you upgrade. Changes that you make with the Format editor to these visualizations override the rangemap configurations. Going forward, generate new single value visualizations by using a query that does not contain the rangemap command, and then use the Format editor to configure ranges, colors, or any additional settings.

Any changes that you make with the editor to single-value visualizations that were generated with = rangemap override edits that you make to the range map command. Additionally, while the editor attempts to preserve the existing configuration, it no longer recognizes rangemap as a valid command to generate these types of visualizations.

Splunk Enterprise now limits the addition of search peers with a large time skew

When you upgrade to Splunk Enterprise 6.4, you will no longer be able to use Splunk Web to add search peers with a time skew of more than 10 minutes from the search head where you are adding the peers.

You can change this setting by editing limits.conf on the search head and setting the addpeer_skew_limit to a positive integer that is lower than its default of 600 (seconds).

Splunk Enterprise support for running multiple searches on a single process could increase memory usage

As of version 6.4, Splunk Enterprise can launch multiple searches on a single process on *nix hosts.

When you upgrade, you should see improved search performance, but you might also see increased memory usage.

This change is not applicable on Windows instances of Splunk Enterprise.

Support for the Deployment Monitor app has been removed

Support for the Splunk Deployment Monitor App has been removed. When you upgrade to Splunk Enterprise 6.4, use the Distributed Management Console (DMC) instead to monitor your distributed deployment. See the Distributed Management Console manual.

This change was introduced in Splunk Enterprise 6.3, but we retain it here for those who upgrade to 6.4 from earlier versions.

Data block signing has been removed

Data block signing has been removed from Splunk Enterprise. The feature has been deprecated for some time.

This change was introduced in Splunk Enterprise 6.3, but we retain it here for those who upgrade to 6.4 from earlier versions.

Accelerated custom data model summaries will rebuild on upgrade

When you upgrade to Splunk Enterprise 6.4, any accelerated custom data model summaries that are present on the instance - such as those created by the Splunk App for Enterprise Security - will be rebuilt. This is because of optimizations to data model searches that have been made, which make the searches incompatible with previously generated summaries.

During the rebuild process, CPU, memory, and disk I/O usage on indexers with the summaries will increase significantly. Searches that rely on those data model summaries will be very slow and might not work fully.

If you need to prevent Splunk Enterprise from automatically rebuilding these summaries on upgrade, make the following changes to your Splunk Enterprise configuration before starting an upgrade:

In datamodels.conf:

acceleration.manual_rebuilds = true

In limits.conf:

[tstats]
allow_old_summaries = true

This change was introduced in Splunk Enterprise 6.3, but can happen on upgrades from 6.3 to 6.4 as well. We retain it here for those who upgrade to 6.4 from earlier versions.

There is now a limit on the number of learned source types

For all versions of Splunk Enterprise, the number of source types that an instance can learn in the process of monitoring and indexing files has been limited.

To reduce instances where CPU and memory usage spiked during such operations, a new attribute that controls how many source types an instance learns when it monitors files and analyzes file contents has been created. The limit is 1000, and you can change this setting by editing the following attribute in limits.conf and restarting Splunk Enterprise:

learned_sourcetypes_limit = <number>

While this setting should prevent memory and CPU spikes, continue to use props.conf and inputs.conf to define and apply source types.

This change was introduced in Splunk Enterprise 6.3, but we retain it here for those who upgrade to 6.4 from earlier versions.

Parallel summarization for data model summaries has been enabled

The number of searches that the Splunk platform runs at a time to generate summary files for data models has changed.

When you upgrade to Splunk Enterprise 6.4, the software runs two concurrent search jobs to generate the summary files, instead of one. This change is called "parallel summarization." It might result in an increase in CPU and memory usage on the instance that contains the data models while the search jobs run, but results in faster availability of data model summaries.

You can change this setting back to the previous default for individual data models. See Parallel summarization in the Knowledge Manager Manual.

This change was introduced in Splunk Enterprise 6.3, but we retain it here for those who upgrade to 6.4 from earlier versions.

You must now enable access to Splunk Enterprise debugging endpoints

Splunk Enterprise used to allow access to debugging endpoints by default. This is no longer the case. When you upgrade, you won't be able to access the debugging endpoints until you make a change in web.conf and restart Splunk Enterprise:

[settings]
enableWebDebug = true

This change was introduced in Splunk Enterprise 6.3, but we retain it here for those who upgrade to 6.4 from earlier versions.

Migration from search head pooling to search head clustering

If you want to migrate to search head clustering from a standalone search head, or from search head pooling, which has been deprecated, you must follow specific instructions and use new Splunk Enterprise instances for search head cluster members. See the following topics in the Distributed Search manual for more information on migrating to search head clustering:

Search head clusters now respect user- and role-based search quotas

When you upgrade to Splunk Enterprise 6.4, any search head clusters that you have deployed will respect and enforce search quotas that are in place for users and roles. This might result in some searches not executing, depending on the number of concurrent searches that are active. To defeat this feature, set the following attributes in limits.conf:

shc_role_quota_enforcement = false
shc_local_quota_check = true

This change was introduced in Splunk Enterprise 6.3, but we retain it here for those who upgrade to 6.4 from earlier versions.

The new App Key Value Store service might increase disk space usage

The App Key Value Store (KV Store) service, which provides a way for you to maintain the state of your application by storing and retrieving data within it, might cause an increase in disk usage on the instance, depending on how many apps you run. You can change where the KV Store service puts its data by editing server.conf, and you can restore data used by KV Store with the splunk clean CLI command. See About the app key value store in the Admin manual.

This change was introduced in Splunk Enterprise 6.2, but we retain it here for those who upgrade to 6.4 from earlier versions.

Splunk Enterprise now identifies search commands that could negatively impact performance

In an effort to improve security and performance, some Search Processing Language (SPL) commands have been tagged with a variable that prompts Splunk Enterprise to warn you about performance impact when you use them in a search query. After an upgrade, you might see a warning message that a search that you run has commands that might have risky side effects.

Results for unaccelerated data models now match results from accelerated data models

The way that unaccelerated data models query indexes for events has changed.

These models now query all indexes, rather than just the default index. This means that the number of results you see for unaccelerated data models should now match the number of results you see for accelerated data models.

After you upgrade, you might see more results for an unaccelerated data model than you did prior to upgrading.

This change was introduced in Splunk Enterprise 6.2, but we retain it here for those who upgrade to 6.4 from earlier versions.

New installed services open additional network ports

Splunk Enterprise installs and runs two new services: App Key Value Store and App Server. This opens two network ports by default on the local machine: 8065 (for Appserver) and 8191 (for App Key Value Store.) Make sure any firewall you run on the machine does not block these ports. The App Key Value Store service also starts an additional process, mongod. If needed, you can disable App Key Value Store by editing server.conf and changing the dbPath attribute to a valid path on a file system that the Splunk Enterprise instance can reach. See About the app key value store in the Admin manual.

This change was introduced in Splunk Enterprise 6.2, but we retain it here for those who upgrade to 6.4 from earlier versions.

Formatting for single-value visualizations has changed

The formatting for single-value visualizations has changed in that these visualizations have been redesigned to be as readable as possible from a distance. When you upgrade, dashboards that use these visualizations might be impacted by very large letters or numbers.

To work around the problem, you can either:

  • Make use of the new time context if you show a numeric value that you can query over time.
  • Use Simple XML to reduce the single value panel height from its default of 115 pixels. Or,
  • Replace the single value panel with a custom HTML panel.

See this post on Splunk Answers for additional information prior to upgrading.

This change was introduced in Splunk Enterprise 6.3, but we retain it here for those who upgrade to 6.4 from earlier versions.

New default values for some attributes can impact Splunk operations over SSL

There are new defaults which can possibly impact running Splunk Enterprise over SSL:

  • The supportSSLv3Only attribute, which controls how Splunk Enterprise handles SSL clients, now has a default setting of true. This means that only clients who can speak the SSL v3 protocol can connect to the Splunk Enterprise instance.
  • The cipherSuite attribute, which controls the encryption protocols that can be used during an SSL connection, now has a default setting of TLSV1+HIGH:@STRENGTH. This means that only clients that possess a Transport Layer Security (TLS) v1 cipher with a 'high' encryption suite can connect to a Splunk Enterprise instance.

This change was introduced in Splunk Enterprise 6.3, but we retain it here for those who upgrade to 6.4 from earlier versions.

Login page customization is no longer available

Login page customization is no longer available as of version 6.2 of Splunk Enterprise. You can only modify the footer of the login page after an upgrade.

Windows-specific changes

Support for Internet Explorer versions 9 and 10 has been removed

Microsoft has announced that support for all versions of Internet Explorer below version 11 has ended as of January 12, 2016. Owing to that announcement, Splunk has ended support for Splunk Web for these same versions. This might result in a suboptimal browsing experience in Internet

When you upgrade, you should also upgrade the version of Internet Explorer that you use to 11 or later. An alternative is to use another browser that Splunk supports.

The Windows host monitoring input no longer monitors application state

The Windows host monitor input has been modified to no longer monitor the state of installed applications.

Due to a bug in the system call that Splunk Enterprise uses to monitor application state, the Windows Installer service attempts to reconfigure all installed applications.

When you upgrade, any Windows host monitoring input stanzas that reference the "Application" attribute no longer function. The host monitoring input continues to function with other host monitoring stanza types that you have defined.

To get application state data, use the Windows Event Log monitor and search for Event ID Nos. 11707 (for installation) or 11724 (for uninstallation/removal.) You can also use a PowerShell script or the Windows Management Instrumentation Command-line tool (WMIC) as follows:

  • PowerShell: Get-WmiObject -Class Win32_Product | Format-List -Property Name,InstallDate,InstallLocation,PackageCache,Vendor,Version,IdentifyingNum
  • WMIC: wmic product get name,version,installdate

This change was introduced in Splunk Enterprise 6.3, but we retain it here for those who upgrade to 6.4 from earlier versions.

New installation and upgrade procedures

The Windows version of Splunk Enterprise has a more streamlined installation and upgrade workflow. The installer now assumes specific defaults (for new installations) and retains existing settings (for upgrades) by default. To make any changes from the default on installations, you must check the "Customize options" button. During upgrades, your only option is to accept the license agreement. See Installation options."

This feature was introduced in Splunk Enterprise 6.2, but we retain it here for those who upgrade to 6.4 from earlier versions.

Changes have been made to support more granular authorization for Windows inputs

Splunk Enterprise has been updated to allow for more control when using Windows inputs like Network Monitoring and Host Monitoring. If you use Splunk Enterprise as a user with a role that does not inherit from other roles, it is possible that the user might not be able to access certain Windows inputs.

The Splunk Web service installs but does not run

The splunkd service handles all Splunk Web operations. However, on Windows instances, the installer still installs the splunkweb service, although the service quits immediately on launch when operating in normal mode. You can configure the service to run in legacy mode by changing a configuration parameter in web.conf. See Start Splunk Enterprise on Windows in legacy mode in the Admin manual.

Important: Do not run Splunk Web in legacy mode permanently. Use legacy mode to temporarily work around issues introduced by the new integration of the user interface with the main splunkd service. Once you correct the issues, return Splunk Web to normal mode as soon as possible.

This change was introduced in Splunk Enterprise 6.2, but we retain it here for those who upgrade to 6.4 from earlier versions.

No support for enabling Federal Information Processing Standards (FIPS) after an upgrade

There is no supported upgrade path from a Splunk Enterprise system with enabled Secure Sockets Layer (SSL) certificates to a system with FIPS enabled. If you need to enable FIPS, you must do so on a new installation.

The default behavior for translating security identifiers (SID) and globally unique identifiers (GUIDs) when monitoring Windows Event Log data has changed

The etc_resolve_ad_obj attribute, which controls whether or not Splunk Enterprise attempts to resolve SIDs and GUIDs when it monitors event log channels, is now disabled by default for all channels. When you upgrade, any inputs.conf monitor stanzas that do not explicitly define this attribute will no longer perform this translation.

This change was introduced in Splunk Enterprise 6.2, but we retain it here for those who upgrade to 6.4 from earlier versions.

Learn about known upgrade issues

To learn about any additional upgrade issues for Splunk Enterprise, see the Known Issues - Upgrade Issues page in the Release Notes.

PREVIOUS
How to upgrade Splunk Enterprise
  NEXT
How to upgrade a distributed Splunk Enterprise environment

This documentation applies to the following versions of Splunk® Enterprise: 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters