Extended example: deploy several standard forwarders

Extended example: deploy several standard forwarders

The following is a configuration that will set up the server MyServer-WIN1 to forward Windows Event Logs to MyServer-SPLUNK1 (10.1.2.2) and MyServer-LINUX1 and MyServer-LINUX2 to forward /var/log/messages to MyServer-SPLUNK2 (10.1.2.3) by configuring the forwarders to be clients of the deployment server on MyServer-SPLUNK3 (10.1.2.4). This assumes that MyServer-SPLUNK1 and MyServer-SPLUNK2 are listening for data from Splunk forwarders on the default port 9997:

1. On MyServer-SPLUNK3, create /opt/splunk/etc/system/local/serverclass.conf with the following settings:

    [global]
    whitelist.0=*

    [serverClass:MyServer-WIN]
    whitelist.0=MyServer-WIN*

    [serverClass:MyServer-WIN:app:winevt]
    stateOnClient=enabled
    restartSplunkd=true

    [serverClass:MyServer-WIN:app:fwd_to_splunk1]
    stateOnClient=enabled
    restartSplunkd=true
    
    [serverClass:MyServer-LINUX]
    whitelist.0=MyServer-LINUX*
    stateOnClient=enabled
    restartSplunkd=true

    [serverClass:MyServer-LINUX:app:linmess]
    stateOnClient=enabled
    restartSplunkd=true

    [serverClass:MyServer-LINUX:app:fwd_to_splunk2]
    stateOnClient=enabled
    restartSplunkd=true

2. On MyServer-SPLUNK3, run the following commands:

mkdir –p /opt/splunk/etc/deployment-apps/fwd_to_splunk1/default 
mkdir –p /opt/splunk/etc/deployment-apps/fwd_to_splunk2/default 
mkdir –p /opt/splunk/etc/deployment-apps/winevt/default 
mkdir –p /opt/splunk/etc/deployment-apps/linmess/default

3. On MyServer-SPLUNK3, make /opt/splunk/etc/deployment-apps/fwd_to_splunk1/default/outputs.conf with the following settings:

    [tcpout]
    defaultGroup=splunk1

    [tcpout:splunk1]
    server=10.1.2.2:9997

4. On MyServer-SPLUNK3, make /opt/splunk/etc/deployment-apps/fwd_to_splunk2/default/outputs.conf with the following settings:

    [tcpout]
    defaultGroup=splunk2

    [tcpout:splunk2]
    server=10.1.2.3:9997

5. On MyServer-SPLUNK3, make /opt/splunk/etc/deployment-apps/winevt/default/inputs.conf with the following settings:

    [WinEventLog:Application]
    disabled=0

    [WinEventLog:Security]
    disabled=0

    [WinEventLog:System]
    disabled=0

6. On MyServer-SPLUNK3, make /opt/splunk/etc/deployment-apps/linmess/default/inputs.conf with the following settings:

    [monitor:///var/log/messages]
    disabled=false
    followTail=1
    sourcetype=syslog

7. Restart Splunk on MyServer-SPLUNK3

8. On MyServer-WIN1, MyServer-LINUX1, and MyServer-LINUX2, install Splunk and create /opt/splunk/etc/system/local/deploymentclient.conf with the following settings:

   [deploymentClient]

   [target-broker:deploymentServer]
   targetUri= 10.1.2.4:8089 

9. Restart Splunk on MyServer-WIN1, MyServer-LINUX1, and MyServer-LINUX2.

After a short delay, Windows Event Logs will begin flowing from MyServer-WIN1 to MyServer-SPLUNK1 and /var/log/messages from MyServer-LINUX1 and MyServer-LINUX2 will start flowing to MyServer-SPLUNK2.

What the communication between the deployment server and its clients looks like

Using the above example, the communication from MyServer-WIN1 to MyServer-SPLUNK3 on port 8089 would look as follows:

MyServer-WIN1: Hello, I am MyServer-WIN1.

MyServer-SPLUNK3: Hello, MyServer-WIN1. I have been expecting to hear from you. I have you down as a member of the MyServer-WIN server class, and you should have the fwd_to_splunk1 (checksum=12345) and winevt (checksum=12378) Apps.

MyServer-WIN1: Hmmm, I don’t have those configs. Using this connection I just opened up to you, can I grab the configs from you?

MyServer-SPLUNK3: Sure! I have them ready for you.

MyServer-WIN1: Thanks! I am going to back off a random number of seconds between 1 and 60 (in case you have a lot of clients that are polling you at the moment) ... Ok, now send me the files.

MyServer-SPLUNK3: Done! You now have fwd_to_splunk1-timestamp.bundle and winevt-timestamp.bundle.

MyServer-WIN1: Awesome! I am going to store them in my $SPLUNK_HOME/etc/apps directory. Now I am going to restart myself, and when I come back up I am going to read the configurations that you sent me directly out of the .bundle files, which I know are just tar balls with a different extension.

A couple of minutes go by.

MyServer-WIN1: Hello, I am MyServer-WIN1.

MyServer-SPLUNK3: Hello, MyServer-WIN1. I have been expecting to hear from you. I have you down as a member of the MyServer-WIN server class, and you should have the fwd_to_splunk1 (checksum=12345) and winevt (checksum=12378) Apps.

MyServer-WIN1: Hmmm, I already have both of those, but thanks anyway!

Later on, an admin modifies the winevt/inputs.conf file on MyServer-SPLUNK3 to disable the collection of System Event Logs, and then runs the CLI command splunk reload deploy-server to force the deployment server to rescan serverclass.conf and the App directories. The next time MyServer-WIN1 talks to MyServer-SPLUNK3, it goes like this:

MyServer-WIN1: Hello, I am MyServer-WIN1.

MyServer-SPLUNK3: Hello, MyServer-WIN1. I have been expecting to hear from you. I have you down as a member of the MyServer-WIN server class, and you should have the fwd_to_splunk1 (checksum=12345) and winevt (checksum=13299) Apps.

MyServer-WIN1: Hmmm, I know I have those configs, but the checksum I have for the winevt configs is different than the one you just told me about. Using this connection I just opened up to you, can I grab the updated winevt config from you?

MyServer-SPLUNK3: Sure! I have it ready for you.

MyServer-WIN1: Thanks! I am going to back off a random number of seconds between 1 and 60 (in case you have a lot of clients that are polling you at the moment) ... Ok, now send me the updated config.

MyServer-SPLUNK3: Done! You now have winevt-newer_timestamp.bundle.

MyServer-WIN1: Awesome! I am going to store it my $SPLUNK_HOME/etc/apps directory and move the old winevt.bundle I had out of the way. Now I am going to restart myself, and when I come back up I am going to have the most up to date config.

• Was this topic useful?
• Post a comment