Splunk® Cloud Services

SPL2 Search Reference

Acrobat logo Download manual as PDF


Acrobat logo Download topic as PDF

dedup command overview

Removes the events that contain an identical combination of values for the fields that you specify.

With the dedup command, you can specify the number of duplicate events to keep for each value of a single field, or for each combination of values among several fields.

Events returned by the dedup command are based on search order. For historical searches, the most recent events are searched first. For real-time searches, the first events that are received are searched, which are not necessarily the most recent events.

Syntax

The required syntax is in bold.

dedup
[<int>]
[keepempty=<bool>]
[consecutive=<bool>]
<field-list>


How the dedup command works

Suppose that you have the following search results:

host action client_ip
www1 view 211.166.11.101
www2 addtocart 194.215.205.19
www3 view 74.53.23.135
www1 addtocart 128.241.220.82
www1 purchase 64.66.0.20
www3 view 107.3.146.207
www2 remove 194.215.205.19

You want to remove search results where the host is a duplicate value.

... | dedup host


The results show the unique host names.

host action client_ip
www1 view 211.166.11.101
www2 addtocart 194.215.205.19
www3 view 74.53.23.135

This example returns only one result for each host value.


You can specify more than one field with the dedup command. For example:

... | dedup host, client_ip

For each combination of host name and client IP address, duplicate results are removed.

host action client_ip
www1 view 211.166.11.101
www2 addtocart 194.215.205.19
www3 view 74.53.23.135
www1 addtocart 128.241.220.82
www1 purchase 64.66.0.20
www3 view 107.3.146.207

In this example, the result with host=www2 and client_ip=194.215.205.19 is removed.

See also

dedup command
dedup command syntax details
dedup command usage
dedup command examples
Last modified on 31 October, 2020
PREVIOUS
branch command examples
  NEXT
dedup command syntax details

This documentation applies to the following versions of Splunk® Cloud Services: current


Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters