Add a sourcetype

Splunk Analytics for Hadoop reaches End of Life on January 31, 2025.

AFter you set up your providers and indexes, you can configure Splunk Analytics for Hadoop to search your virtual indexes by sourcetype.

Though most source types are log formats, any common data input format can be a source type. If your data is unusual, you might need to create a source type with customized event processing settings. And if your data source contains heterogeneous data, you might need to assign the source type on a per-event (rather than a per-source) basis.

See Why sourcetypes matter in the Splunk Enterprise documentation to learn more about why you might want to use sourcetyping on your HDFS data.

To add a sourcetype to an HDFS data source, you add a stanza to $SPLUNK_HOME/etc/system/local/props.conf. When defining sourcetypes for HDFS data, keep in mind that searches of HDFS data occur at search-time, not index time and that Hunk only reads the latest timestamps and not original HDFS timestamps. As a result, timestamp recognition may not always works as expected.

In the example below, we add two sourcetypes. A new sourcetype access_combined represents data from the access_combined log files. mysqld will let you search data from the specified mysqld.log file(s):

[source::.../access_combined.log]
sourcetype=access_combined
priority=100

[source::.../mysqld.log]
sourcetype=mysqld
priority=100

(You do not need to restart)

For information about searching, including searching by sourcetypes, see Use fields to search in the Splunk Enterprise Search Tutorial.

Note the following when adding a sourcetype:

• INDEXED_TIME extractions do not work with Splunk Analytics for Hadoop.
• While search time extractions should work with Splunk Analytics for Hadoop, it's easier to use the SimpleCSVRecordReader to do what you're looking for (if the file has a header) by adding it to the default list:

#append the SimpleCSVRecordReader to the default list:
vix.splunk.search.recordreader = ...,com.splunk.mr.input.SimpleCSVRecordReader 
vix.splunk.search.recordreader.csv.regex = <a regex to match csv files>
vix.splunk.search.recordreader.csv.dialect = tsv