Use Splunk for application management

Use Splunk for application management

Every deployment is different -- a different combination of infrastructure, tools, processes -- and Splunk makes the assumption that you are the ultimate expert on your data. On a fundamental design level, Splunk is focused on providing tools that you can tailor to your environment and your needs.

• Data-agnostic indexing gives you a closer, more flexible look at your data through aggregation and cross-tier searching.
• Splunk knowledge helps you build on and expand your existing expertise by letting you add structure your data in a smart focused way.
• Administrative tools let you optimize your Splunk deployment and performance.

Regardless of your use case, the process is similar.

Identify data sources

First, define the pieces of your application and its environment and see where there is data that might be useful in Splunk. In the interconnected world of application management, everything has an effect, and in the long run, logs are only one source of useful data. A large percentage of the time, problems in your application environment are not due to problems with your application or your application servers. What components in your environment do you care about? Where do they provide data that you can use in Splunk? All of the following are possible sources of useful data for Splunk:

• Endpoints - e.g., http://myapp.com for browser clients, http://widgets.myapp.com for ajax clients
• Infrastructure software, such as web servers and application servers
• Custom components, such as web applications, application databases, etc.
• Server hardware, which you can monitor for metrics such as storage, CPU, etc.
• Network hardware, such as routers and firewalls
• Network services & connectivity, such as DNS, LDAP, etc.
• Legacy and external systems, such as applications running on a mainframe

Eat data

Once you have an idea of where you want to start, you can bring data into Splunk and begin to create Splunk knowledge. Splunk's flexible model means that "bringing data into Splunk" is typically a two-step process. You start by getting Splunk to "eat" the data -- to bring it into a Splunk index and enhance it with the basic structure Splunk needs to search and display. This is enough to start cross-tier search and discovery and can be incredibly useful on its own. At any time, you can also add structure and knowledge to data that is already in Splunk to make your searches more powerful, and easier to use and share.

Splunk can eat any kind of timestamped data. Some common data types you can eat with Splunk are:

• logs
• configuration: You can eat configuration data from a database, monitor changes to the contents of configuration files, or monitor configuration changes detected at the level of the operating system.
• metrics: You can import data from most tools that provide metrics or alerts.
• scripted inputs: Splunk can accept events from scripts that you provide. Scripted inputs can be used to get data from a number of sources, including command-line tools such as vmstat or netstat, APIs and other remote data interfaces, virtualization layer metrics and logs, and message queues. See Set up scripted inputs in the Admin manual for more information.

You do not have to eat all your data at once. Bringing data into Splunk is usually an incremental process. As you learn more about your application, you can:

• Get more data: When you are troubleshooting, Splunk is like time-travel -- it lets you go back and see the state of your systems before the problem occurred. When you are no longer putting out fires and have time to look deeper, Splunk helps you uncover relationships you didn't know were there. But you can't discover relationships or troubleshoot a problem unless you have the data. When a system goes down, or a major problem occurs, it is too late to gather the data that preceded the symptom.
• Throw away genuinely uninteresting data. Sometimes your logs contain bits that have no value to Splunk, such as embedded binary files. You can filter out this data before it gets to the Splunk index.

Structure data

Once your data has been indexed, Splunk provides tools to interpret, classify, enrich and normalize it.

• Add structure to the data using techniques for labeling and grouping events, fields, event types, tags, etc.
• Ensure your dashboards and searches will scale using techniques such as summary indexes.

Use data

Splunk has many tools that allow you to search, analyze, and present your data in many ways, regardless of use case:

• Create sophisticated searches using commands like transaction, append, and dedup.
• Analyze your data using statistical and charting commands like stats, chart, and timechart.
• Visualize your data with custom reports, dashboards, and alerts.
• Make your searches repeatable and report on your data with saved searches, reports, PDFs, dashboards, and alerts.
• Speed up your reports and dashboards with summary indexing.
• Set up Users and Roles to control who sees what data and to distribute your saved searches and dashboards.

Rinse and repeat

As you use Splunk and find where it has value to you, you will add more data and structure. You can do this incrementally at any time:

• Identify and eat more data you want to use.
• Ask your developers to log more data.
• Find new ways to use the data.

• Was this topic useful?
• Post a comment