Define a geospatial lookup in Splunk Web

Use geospatial lookups to create queries that return results that Splunk software can use to generate a choropleth map visualization. Choropleth maps cannot be rendered without the data generated by corresponding geospatial lookups.

A geospatial lookup matches location coordinates in your events to location coordinate ranges in a geographic feature collection known as a Keyhole Markup Zipped (KMZ) or Keyhole Markup Language (KML) file and outputs fields to your events that provide corresponding geographic feature information that is encoded in the feature collection. This information represents a geographic region that shares borders with geographic regions of the same type, such as a country, state, province, or county.

Splunk software provides two geospatial lookups that enable you to render choropleth maps at two levels of granularity:

• The USA, divided into states
• The world, divided into countries

This topic shows you how to create additional geospatial lookups that break choropleth maps into other types of regions, such as counties, provinces, timezones, and so on.

For information about choropleth maps and geographic data visualizations, see Mapping data in the Dashboards and Visualizations manual.

The workflow to create a geospatial lookup in Splunk Web is to upload a file, share the lookup table file, and then create the lookup definition from the lookup table file. If you're using Splunk Enterprise, you can also define geospatial lookups using configuration files. See Configure geospatial lookups for details.

Your role must have the upload_lookup_files capability. Without it you cannot create or edit geospatial lookups in Splunk Web.

See Define roles with capabilities in Securing Splunk Enterprise.

The FeatureId and featureCollection fields

Geospatial lookups differ from other lookup types in that they are designed to output these two fields: featureId and featureCollection. The featureId is the name of the feature, such as California, CA, or whatever name is encoded in the feature collection. The featureCollection field provides the name of the lookup in which the feature was found.

If you pipe the output of a geospatial lookup into a geom command, the command does not need to be given the lookup name. The geom command detects the featureId and featureCollection fields in the event and uses the lookup to generate the geographic data structures that Splunk software requires to generate a choropleth map. However, geographic data structures can be large. It is strongly discouraged to pipe events into the geom command, because geographic data structures are attached to every event. Instead, first perform stats on the results of your geographic lookup, and only perform geom on an aggregated statistic like count by featureId.

The Feature Id Element field

The Feature Id Element field is an XPath expression that defines a path from a <Placemark> element in the KML file to an XML element that contains the name of the <Placemark> element. A typical <Placemark> element associates a name element to one or more <Polygon> elements. Each Placemark element is considered a geographic feature, and Splunk software uses its name as the unique Feature Id value that matches the geographic feature to an event in your data.

The default Feature Id Element setting is /Placemark/name, because the name element is typically tagged with <name> and located immediately beneath <Placemark> in the XML hierarchy. If the KML file you are using has a different architecture, provide an XPath expression for Feature Id Element that indicates where the <name> element is located relative to the <Placemark> element.

• The Feature Id Element field may be required in cases where the featureID field generated by the lookup is an empty string, or when the feature collection returns incorrect features by default. In the latter case, the feature may be a peer of the default feature or be located relative to the default feature.
• To determine what path you need, review the geographic feature collection. Each feature in the collection is tagged with <Placemark>, and each <Placemark> contains a name that the lookup includes as a featureId field in associated events.

XPath and feature id element example

The following is an example <Placemark> element extracted from a KML file.

<Placemark>
<name>Bayview Park</name>
<visibility>0</visibility>
<styleUrl>#msn_ylw-pushpin15</styleUrl>
<Polygon>
<tessellate>1</tessellate>
<outerBoundaryIs>
<LinearRing>
<coordinates>
-122.3910323868129,37.70819686392956,0 -122.3902274700583,37.71036559447013,0 -122.3885849520798,37.71048623150828,0 -122.38693857563,37.71105170319798,0 -122.3871609118563,37.71133850017815,0 -122.3878019922009,37.71211354629052,0 -122.3879259342663,37.7124647025671,0 -122.3880447162415,37.71294302956619,0 -122.3881688500638,37.71332710079798,0 -122.3883793249067,37.71393746998403,0 -122.3885493512011,37.71421419664032,0 -122.3889255249081,37.71472750657607,0 -122.3887475583787,37.71471048572742,0 -122.3908349856662,37.71698679132378,0 -122.3910091960123,37.71714685034637,0 -122.3935812625442,37.71844151854729,0 -122.3937942835165,37.71824640920892,0 -122.3943907534492,37.71841931125917,0 -122.3946363652554,37.71820562249533,0 -122.3945665820268,37.71790603321808,0 -122.3949430786581,37.71764553372926,0 -122.3953167478058,37.71742547689517,0 -122.3958076264322,37.71693521458138,0 -122.3960283880498,37.7166859403894,0 -122.3987339294558,37.71607634724589,0 -122.3964526840739,37.71310454861037,0 -122.396237742007,37.71265453835174,0 -122.3959650878797,37.7123218368849,0 -122.3955644372275,37.71122536767665,0 -122.3949262649838,37.7082082656386,0 -122.3910323868129,37.70819686392956,0 
</coordinates>
</LinearRing>
</outerBoundaryIs>
</Polygon>
</Placemark>

Let's take a look at another <Placemark> element extracted from a KML file.

<Placemark> 
<name>MyFeature</name>
<ExtendedData> 
<SchemaData> 
<SimpleData name="placename">foo</SimpleData> 
<SimpleData name="bar">baz</SimpleData> 
</SchemaData> 
...

The XPath expression for this <Placemark> fragment would be feature_id_element=/Placemark/ExtendedData/SchemaData/SimpleData[@name='placename'].

Upload the lookup table file

To use a lookup table file, you must upload the file to your Splunk platform.

Prerequisites

• About lookups for information on lookups.
• An available .kmz or .kml table file.
• Your role must have the upload_lookup_files capability. Without it you cannot upload lookup table files in Splunk Web. See Define roles with capabilities in Securing Splunk Enterprise.

Steps

1. Select Settings > Lookups.
2. In the Actions column, click Add new next to Lookup table files.
3. Select a Destination app from the list.
   Your lookup table file is saved in the directory where the application resides. For example: $SPLUNK_HOME/etc/users/<username>/<app_name>/lookups/.
4. Click Choose File.
5. Enter the destination file name. This is the name the lookup table file will have on the Splunk server. Use a file name ending in ".kmz" or ".kml".
6. Click Save.

Share the lookup table file

After you upload the lookup file, tell the Splunk software which applications can use this file. The default app is Launcher.

1. Select Settings > Lookups.
2. From the Lookup manager, click on Lookup table files.
3. Click Permissions under Sharing for the file you want to share.
4. If you want the lookup to be available globally, select Global. If you want the lookup to be specific to this app only, select This app only.
5. Click Save.

Create the lookup definition

You must create a lookup definition from the lookup table file.

Prerequisites
In order to create the lookup definition, share the lookup table file so that Splunk software can see it.

Review

• About lookups and field actions.
• Configure a time-bounded lookup.
• Make your lookup automatic.
• The feature_id_element field.

Steps

1. Select Settings > Lookups.
2. Click Lookup definitions.
3. Click New to add a new definition.
4. Select a Destination app from the list.
   Your lookup table file is saved in the directory where the application resides. For example: $SPLUNK_HOME/etc/users/<username>/<app_name>/lookups/.
5. Give your lookup definition a unique Name.
6. Select Geospatial for the lookup Type.
7. Select the Lookup file.
8. (Optional) To define advanced options for your lookup, select the Advanced options check box.

   Advanced options	Description
   Feature Id Element	An XPath expression that defines a path from a Polygon element in the KML file to another XML element or attribute that contains the name of the feature. Required when named Placemark elements are not in use.

9. Click Save.

Your lookup is defined as a geospatial lookup and appears in the list of Lookup definitions.

Share the lookup definition

Now that you have created the lookup definition, you need to specify in which apps you want to use the definition.

1. In the Lookup definitions list, for the lookup definition you created, click Permissions.
2. In the Permissions dialog box, under Object should appear in, select All apps to share globally or the app that you want to share it with.
3. Click Save.

   In the Lookup definitions page, your lookup now has the permissions you have set.

Permissions for lookup table files must be the same or larger than those of the lookup definitions that use those files.

You can use this field lookup to add information from the lookup table file to your events. You can use the field lookup by specifying the lookup command in a search string. Or, you can set the field lookup to run automatically.

Make the lookup automatic

Instead of using the lookup command in your search when you want to apply a field lookup to your events, you can set the lookup to run automatically. See Define an automatic lookup for more information.

Search commands and geospatial lookups

If you have created a geospatial lookup definition, you can interact with geospatial lookup through the inputlookup search command. You can use inputlookup to show all geographic features on a choropleth map visualization.

This example uses the default geo_us_states lookup.

Prerequisites

• A geospatial lookup, see above.

Steps:

1. From the Search and Reporting app, use the inputlookup command to search on the contents of your geospatial lookup.

   | inputlookup geo_us_states

2. Click on the Visualization tab.
3. Click on Cluster Map and select Chloropleth Map for your visualization.

A chloropleth map displaying the featureIds of your geospatial lookup appears. For more information on chloropleth maps, see Generate a chloropleth map in the Dashboards and Visualizations manual.

Configure geospatial lookups with .conf files

Geospatial lookups can also be configured using .conf files. See Configure geospatial store lookups for more information.