Using logs for applications is a good way to identify errors, threats, profiling and many more critical facts. That is why developers prefer Fluent Bit. If we can analyse, query and visualise data in a meaningful manner, then the results will be more productive. That’s where Splunk comes in. In this post, I concentrate on how to have these two magnificent tools link in order to get the optimum outcome.

First I’ll explain the environment that this setup was tested on. 

• Splunk Enterprise: Version 8.0.1
• Fluent Bit: Version 1.6.10
• Fluent Bit running OS version: Ubuntu 20.04.1 LTS

Splunk is a software for searching, monitoring, and analysing machine-generated big data via a Web-style interface. First we will take a look at the configuration that has to be done in Splunk. I hope you have a basic understanding of Splunk-related terminologies that are used here.

Let’s start with creating an Index in Splunk.

• Log in to your Splunk Enterprise application and click on the Settings menu item in the upper right corner where you can find the Indexes option.

 

 

• Click on the New Index button at the upper right corner, where you get a pop up window like below.

 

 

 

• Provide a name to your Index and keep other fields as it is if your are new to Splunk. Otherwise you can customise those values according to your need but that part is irrelevant to the purpose of this post. 

• After saving the above pop up, we need to create an HEC (HTTP Event Collector) token.

• To navigate to that window, click on Settings like earlier and then select Data inputs.

 

 

 

• Click on the New Token button in the upper right hand corner where you will get four steps to fill data.

 

First - Provide a name 

 

 

 

 

 

 

 

Now we are good to go ahead with Fluent Bit related changes. It’s all about the configuration file.

 

[INPUT]
    Name    dummy

[FILTER]
    Name    nest
    Match   *
    Operation   nest
    Wildcard    *
    Nest_under  event

[FILTER]
    Name      modify
    Match     *
    Add index flb_index

[OUTPUT]
    Name        splunk
    Match       *
    Host        <your splunk server ip goes here>
    Port        8088
    TLS         Off
    TLS.Verify  Off
    Splunk_Token <your splunk HEC token goes here>
    Splunk_Send_Raw On

 

I am not going to describe the configuration parameter included here, because it has been clearly explained in the Fluent Bit documentation. 

Once you run the Fluent Bit configuration file, you will be able to query the index from Splunk application.

 

 

Results will look something like below.

 

 

 

Done! It is quite straightforward and easy as that. Hope you will also find it easy too! 

 

---

---