Splunk® Supported Add-ons

Splunk Add-on for Apache Web Server

Configure enhanced log formatting on the Apache Web Server using httpd.conf

Configure log formatting on the Apache Web Server using httpd.conf. This lets the Splunk Add-on for Apache Web Server monitor your deployment's log files.

To make sure you have all the required fields present in apache access_log, Splunk best practice is to use an enhanced custom log format in the apache httpd.conf file. For more information, see Configure monitor inputs for the Splunk Add-on for Apache Web Server.


    The Splunk Add-on for Apache Web Server now supports the default out-of-the-box log format of Apache Web Server; the add-on extracts all possible CIM fields where present and makes them searchable. However, this log format is missing many critical CIM fields (e.g. user, hostname, etc), so note that the add-on does not apply CIM Data Model tags to those events. Splunk best practice is still to use an enhanced custom log format which requires modifications to httpd.conf, and use the apache:access:kv or apache:access:json sourcetypes. To use the default out-of-the-box events, ignore the below steps & proceed with Configure monitor inputs for the Splunk Add-on for Apache Web Server.
  1. Open the httpd.conf in the Apache Web Server installation folder $APACHE_HOME/etc/apache/conf/httpd.conf

    The default location of httpd.conf might be different on different platforms.

  2. Look for the statement <IfModule log_config_module> and replace the whole block with the following stanza:
    <IfModule log_config_module>
        #
        # The following directives define some format nicknames for use with
        # a CustomLog directive (see below).
        #
        LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
        LogFormat "%h %l %u %t \"%r\" %>s %b" common
    
     <IfModule logio_module>
        # You need to enable mod_logio.c to use %I and %O
        LogFormat "time=%{%s}t.%{usec_frac}t, bytes_in=%I, bytes_out=%O, cookie=\"%{Cookie}i\", server=%v, dest_port=%p, http_content_type=\"%{Content-type}i\", http_method=\"%m\", http_referrer=\"%{Referer}i\", http_user_agent=\"%{User-agent}i\", ident=\"%l\", response_time_microseconds=%D, client=%h, status=%>s, uri_path=\"%U\", uri_query=\"%q\", user=\"%u\"" splunk_kv
    
        #LogFormat "{\"time\":\"%{%s}t.%{usec_frac}t\", \"bytes_in\":\"%I\", \"bytes_out\":\"%O\", \"cookie\":\"%{Cookie}i\", \"server\":\"%v\", \"dest_port\":\"%p\", \"http_content_type\":\"%{Content-type}i\", \"http_method\":\"%m\", \"http_referrer\":\"%{Referer}i\", \"http_user_agent\":\"%{User-agent}i\", \"ident\":\"%l\", \"response_time_microseconds\":\"%D\", \"client\":\"%h\", \"status\":\"%>s\", \"uri_path\":\"%U\", \"uri_query\":\"%q\", \"user\":\"%u\"}" splunk_json    
      
      #LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
    
    </IfModule>
        #
        # The location and format of the access logfile (Common Logfile Format).
        # If you do not define any access logfiles within a <VirtualHost>
        # container, they will be logged here.  Contrariwise, if you *do*
        # define per-<VirtualHost> access logfiles, transactions will be
        # logged therein and *not* in this file.
        #
        # CustomLog "logs/access_log" common
        #
        # If you prefer a logfile with access, agent, and referer information
        # (Combined Logfile Format) you can use the following directive.
        #
        CustomLog "logs/access_log" splunk_kv
        #CustomLog "logs/access_log" splunk_json
        #CustomLog "logs/access_log" combined
    </IfModule>
    
  3. Choose either the splunk_kv or splunk_json format for access logs. Only one format can be enabled at a time. The information provided by either splunk_kv and splunk_json is the same. The difference is only in formatting. By default, splunk_kv is enabled and the splunk_json is disabled. To enable the splunk_json format:
    1. Comment out the splunk_kv definition and log file directives.
    2. Uncomment the splunk_json definition and log file directives.
  4. The KV pair formatting is simpler when compared to JSON formatting. The best practice is to include new custom fields, if required, in KV format. If applied in JSON format, then it must comply with strict JSON formatting.

  5. Validate the syntax of the conf file after you make your changes.
    sudo apache2ctl configtest

    or

    sudo httpd -t
    If the output says syntax ok, proceed.
  6. Restart Apache Web Server. If the log format in Apache Web Server is configured correctly, you receive log files that look like this:
    • For splunk_kv
      time=###TIME###.000000, bytes_in=###BYTES_IN###, bytes_out=###BYTES_OUT###, cookie="###COOKIE###", server=C6852495051.domain, dest_port=###DEST_PORT###, http_content_type="###HTTP_CONTENT_TYPE###", http_method="###HTTP_METHOD###", http_referrer="###HTTP_REFERRER###", http_user_agent="###HTTP_USER_AGENT###", ident="1", response_time_microseconds=###RESPONSE_TIME_MICROSECONDS###, client=###CLIENT###, status=###STATUS###, uri_path="###URI_PATH###", uri_query="###URI_QUERY###", user="xyz123"
      
    • For splunk_json
      {"time":"###TIME###.000000", "bytes_in":"###BYTES_IN###", "bytes_out":"###BYTES_OUT###", "cookie":"###COOKIE###", "server":"C6852495051.domain", "dest_port":"###DEST_PORT###", "http_content_type":"###HTTP_CONTENT_TYPE###", "http_method":"###HTTP_METHOD###", "http_referrer":"###HTTP_REFERRER###", "http_user_agent":"###HTTP_USER_AGENT###", "ident":"1", "response_time_microseconds":"###RESPONSE_TIME_MICROSECONDS###", "client":"###CLIENT###", "status":"###STATUS###", "uri_path":"###URI_PATH###", "uri_query":"###URI_QUERY###", "user":"xyz123"}
      
  7. The new fields will be auto extracted without making any change in the add-on's field extraction.

Last modified on 15 June, 2022
Upgrade the Splunk Add-on for Apache Web Server   Configure monitor inputs for the Splunk Add-on for Apache Web Server

This documentation applies to the following versions of Splunk® Supported Add-ons: released


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters