Curate Splunk knowledge with Manager
As your organization uses Splunk, knowledge is added to the base set of event data indexed within it. Searches are saved and scheduled. Tags are added to fields. Event types and transactions that group together sets of events are defined. Lookups and workflow actions are engineered.
The process of knowledge object creation starts out slow, but can get complicated over time. It's easy to reach a point where users are "reinventing the wheel," creating searches that already exist, designing redundant event types, and so on. These things may not be a big issue if your user base is small, but they can cause unnecessary confusion and repetition of effort, especially as they accumulate over time.
This topic discusses how knowledge managers can use Splunk Manager to take charge of the knowledge objects in their Splunk system and show them who's boss. Splunk Manager can give a savvy and attentive knowledge manager a view into what knowledge objects are being created, who they're being created by, and (to some degree) how they are being used.
With Manager, you can easily:
- Create knowledge objects as necessary, either "from scratch" or through object cloning.
- Review knowledge objects as they are created, with an eye towards reducing redundancy, ensuring that naming standards are followed, and that "bad" objects are removed before they develop lots of downstream dependencies.
- Ensure that knowledge objects with relevancy beyond a particular working team, role, or app are made available to other teams, roles, and users of other apps.
- Delete knowledge objects that do not have significant "downstream" dependencies.
Note: This topic assumes that as a knowledge manager you have an admin role or a role with an equivalent permission set.
Using configuration files instead of Manager
In previous releases Splunk users edited Splunk's configuration files directly to add, update, or delete knowledge objects. Now they can use Manager, which provides a user-friendly interface with those very same configuration files.
We do recommend having some familiarity with configuration files. The reasons for this include:
- Some Manager functionality makes more sense if you understand how things work at the configuration file level. This is especially true for the Field extractions and Field transformations pages in Manager.
- Functionality exists for certain knowledge object types that isn't (or isn't yet) expressed in the Manager UI.
- Bulk deletion of obsolete, redundant, or improperly defined knowledge objects is only possible with configuration files.
- You may find that you prefer to work directly with configuration files. For example, if you're a long-time Splunk user, brought up on our configuration file system, it may be the medium in which you've grown accustomed to dealing with knowledge objects. Other users just prefer the level of granularity and control that configuration files can provide.
Wherever you stand with Splunk's configuration files, we want to make sure you can use them when you find it necessary to do so. To that end, you'll find that the Knowledge Manager manual includes instructions for handling various knowledge object types via configuration files. For more information, see the documentation of those types.
For general information about configuration files in Splunk, see the following topics in the Admin manual:
You can find examples of the current configuration
.example files in the "Configuration file reference" chapter of the Admin manual.
Monitor and organize knowledge objects
As a knowledge manager, you should periodically check up on the knowledge object collections in your Splunk implementation. You should be on the lookout for knowledge objects that:
- Fail to adhere to naming standards
- Are duplicates/redundant
- Are worthy of being shared with wider audiences
- Should be disabled or deleted due to obsolescence or poor design
Regular inspection of the knowledge objects in your system will help you detect anomalies that could become problems later on.
Most healthy Splunk implementations end up with a lot of tags, which are used to perform searches on clusters of field/value pairings. Over time, however, it's easy to end up with tags that have similar names but which produce surprisingly dissimilar results. This can lead to considerable confusion and frustration.
Here's a procedure you can follow for curating tags. It can easily be adapted for other types of knowledge objects handled through Manager.
1. Go to Manager > Tags > List by tag name.
2. Look for tags with similar or duplicate names that belong to the same app (or which have been promoted to global availability for all users). For example, you might find a set of tags like
authentications in the same app, where one tag is linked to an entirely different set of field/value pairs than the other.
Alternatively, you may encounter tags with identical names except for the use of capital letters, as in
Crash. Tags are case-sensitive, so Splunk sees them as two separate knowledge objects.
Keep in mind that you may find legitimate tag duplications if you have the App context set to All, where tags belonging to different apps have the same name. This is often permissible--after all, an
authentication tag for the Windows app will have to be associated with an entirely different set of field/value pairs than an
authentication for the UNIX app, for example.
3. Try to disable or delete the duplicate or obsolete tags you find, if your permissions enable you to do so. However, be aware that there may be objects dependent on it that will be affected. If the tag is used in saved searches, dashboard searches, other event types, or transactions, those objects will cease to function once the tag is removed or disabled. This can also happen if the object belongs to one app context, and you attempt to move it to another app context.
For more information, see "Disable or delete knowledge objects," below.
4. If you create a replacement tag with a new, more unique name, ensure that it is connected to the same field/value pairs as the tag that you are replacing.
Using naming conventions to head off object nomenclature issues
If you set up naming conventions for your knowledge objects early in your implementation of Splunk you can avoid some of the thornier object naming issues. For more information, see "Develop naming conventions for knowledge objects" in this manual.
As a Knowledge Manager, you can set knowledge object permissions to restrict or expand access to the variety of knowledge objects within your Splunk implementation.
In some cases you'll determine that certain specialized knowledge objects should only be used by people in a particular role, within a specific app. And in others you'll move to the other side of the scale and make universally useful knowledge objects globally available to all users in all apps. As with all aspects of knowledge management you'll want to carefully consider the implications of these access restrictions and expansions.
When a Splunk user first creates a new saved search, event type, transaction, or similar knowledge object, it is only available to that user. To make that object available to more people, Manager provides the following options, which you can take advantage of if your permissions enable you to do so. You can:
- Make the knowledge object available globally to users of all apps (also referred to as "promoting" an object).
- Make the knowledge object available to all users of an app.
- Restrict (or expand) access to global or app-specific objects by user or role.
- Set read/write permissions at the app level for roles, to enable users to share or delete objects they do not own.
How do permissions affect knowledge object usage?
To illustrate how these choices can affect usage of a knowledge object, imagine that Bob, a user of the (fictional) Network Security app with a "Firewall Manager" role, creates a new event type named
firewallbreach, which finds events that indicate firewall breaches. Here's a series of permissions-related issues that could come up, and the actions and results that would follow:
|When Bob first creates
||Bob updates the permissions of the
||Anyone using Splunk in the Network Security app context can see, work with, and edit the |
|A bit later on, Mary, the knowledge manager, realizes that only users in the Firewall Manager role should have the ability to edit or update the
||Mary restricts the ability to edit the event type to the Firewall Manager role.||Users of the Network Security app can use the |
|At some point a few people who have grown used to using the very handy
||They make their case to the knowledge manager, who promptly promotes the
||Now, everyone that uses this implementation of Splunk can use the |
Note: You may want to set your Splunk implementation up so that only people with Admin-level roles can share and promote knowledge objects. This would make you (and your fellow knowledge managers) gatekeepers with approval capability over the sharing of new knowledge objects.
Permissions - Getting started
To change the permissions for a knowledge object, follow these steps:
1. In Manager, navigate to the page for the type of knowledge object that you want to update permissions for (such as Searches and reports or Event types).
2. Find the knowledge object that you created (use the filtering fields at the top of the page if necessary) and click its Permissions link.
3. On the Permissions page for the knowledge object in question, perform the actions in the following subsections depending on how you'd like to change the object's permissions.
Make an object available to users of all apps
To make an object globally available to users of all apps in your Splunk implementation:
1. Navigate to the Permissions page for the knowledge object (following the instructions above).
2. Under [Knowledge object type] should appear in:, select All apps.
3. In the Permissions section, for Everyone, select a permission of either Read or Write:
- Read enables users to see and use the object, but not update its definition. In other words, when users only have Read permission for a particular saved search, they can see it in the top level navigation (the "Searches & Reports" dropdown, for example) and they can run it. But they can't update the the search string, change its time range, and save their changes.
- Write enables users to view, use, and update the defining details of an object as necessary.
- If neither Read or Write is selected then users cannot see or use the knowledge object.
4. Save the permission change.
Make an object available to users of a particular app
To restrict the usage of a knowledge object to a specific app, you first have to be in the context of that app. To do this, click the App dropdown in the upper right-hand corner of the screen and select the app to which you'd like to restrict the knowledge object.
- If the knowledge object is private, or shared globally, then all you have to do is navigate to the Permissions page for that object and select This app under [Knowledge object type] should appear in:. Then select a permission of either Read or Write for Everyone as appropriate.
- If usage of a knowledge object is already restricted to an app and you want to switch its context to another app, click the Move link (it will only appear if you have sufficient permissions to move it). This will enable you to quickly and easily choose another app context for the knowledge object.
- Keep in mind, however, that switching the app context of an knowledge object can have downstream consequences for objects that have been associated with it. For more information see "Disable or delete knowledge objects", below.
Restrict knowledge object access by role
You can use this method to lock down various knowledge objects from alteration by specific roles. You can arrange things so users in a particular role can use the knowledge object but not update it--or you can set it up so those users cannot see the object at all. In the latter case, the object will not show up for them in Manager, and they will not find any results when they search on it.
If you want restrict the ability to see or update a knowledge object by role, simply navigate to the Permissions page for the object. If you want members of a role to:
- Be able to use the object and update its definition, give that role Read and Write access.
- Be able to use the object but be unable to update it, give that role Read access only (and make sure that Write is unchecked for the Everyone role).
- Be unable to see or use the knowledge object at all, leave Read and Write unchecked for that role (and unchecked for the Everyone role as well).
For more information about role-based permissions in Splunk see "About users and roles" in the Admin manual.
When you use Splunk as delivered the only roles that can set permissions for knowledge objects are Power and Admin. If you want to grant other roles the ability to set permissions for knowledge objects, navigate to Manager > Apps and click Permissions for a specific app to go to its Permissions page. On this page you can determine which roles have read or write access to the knowledge objects that the app contains. Grant a role write access if you wish for it to have full ability to set knowledge object permissions, including the ability to share searches, alerts, and dashboards when these things are created via Splunk Web.
The ability to set permissions for knowledge objects share knowledge objects is controlled at the app level--it is not connected to a capability like other actions such as scheduling searches or changing default input settings.
For detailed information about enabling roles to set permissions for knowledge objects within apps, see "Step 5: Set Permissions" in the Developing Dashboards, Forms, and Views for Splunk Web Manual.
If a Splunk user leaves your team and you need to delete that user or role from the Splunk system, be aware that you will lose any knowledge objects belonging to them that have a sharing status of private. If you want to keep those knowledge objects, share them at the app or global level before deleting the user or role.
Disable or delete knowledge objects
Let's start off by saying that Manager makes it fairly easy to disable or delete knowledge objects as long as your permissions enable you to do so. In Splunk, the ability to delete knowledge objects in Manager really depends on a set of factors:
- You cannot delete default knowledge objects that were delivered with Splunk (or with the App) via Manager. If the knowledge object definition resides in the app's default directory, it can't be removed via Manager. It can only be disabled (by clicking Disable). Only objects that exist in an app's "local" directory are eligible for deletion.
- You can delete knowledge objects that you have created, and which haven't been shared. Once a knowledge object you've created is shared with other users, your ability to delete it is revoked, unless you have write permissions for the app to which they belong (see the next point).
- To delete all other knowledge objects, you need to have write permissions for the application to which they belong. This applies to knowledge objects that are shared globally as well as those that are only shared within an app--all knowledge objects belong to a specific app, no matter how they are shared.
- App-level write permissions are usually only granted to users with admin-equivalent roles.
To sum up: the ability to edit a knowledge object has nothing to do with the ability to delete it. If you can't delete a particular knowledge object you may still be able to disable it, which essentially has the same function as knowledge object deletion without removing it from the system.
Deleting knowledge objects with downstream dependencies
You have to be careful about deleting knowledge objects with downstream dependencies, as this can have negative impacts.
For example, you could have a tag that looks like the duplicate of another, far more common tag. On the surface it would seem to be harmless to delete the dup tag. But what you may not realize is that this duplicate tag also happens to be part of a search that a very popular event type is based upon. And that popular event type is used in two important saved searches--the first is the basis for a well-used dashboard panel, and the other is used to populate a summary index that is used by searches that run several other dashboard panels. So if you delete that tag, the event type breaks, and everything downstream of that event type breaks.
This is why it is important to nip poorly named or defined knowledge objects in the bud, before they become inadvertently hard-wired into the workings of your deployment. The only way to identify the downstream dependencies of a particular knowledge object is to search on it, find out where it is used, and then search on those things to see where they are used--it can take a bit of detective work. There is no "one click" way to bring up a list of knowledge object downstream dependencies at this point.
If you really feel that you have to delete an knowledge object, and you're not sure if you've tracked down and fixed all of its downstream dependencies, you could try disabling it first to see what impact that has. If nothing seems to go seriously awry after a day or so, delete it.
Deleting knowledge objects in configuration files
Note that when you use manager, you can only disable or delete one knowledge object at a time. If you need to remove large numbers of objects, the most efficient way to do it is by removing the knowledge object stanzas directly through the configuration files. Keep in mind that several versions of a particular configuration file can exist within your system. In most cases you should only edit the configuration files in
$SPLUNK_HOME/etc/system/local/, to make local changes on a site-wide basis, or
$SPLUNK_HOME/etc/apps/<App_name>/local/, if you need to make changes that apply only to a specific app.
Do not try to edit configuration files until you have read and understood the following topics in the Admin manual:
Prerequisites for knowledge management
Develop naming conventions for knowledge objects
This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7