> For the complete documentation index, see [llms.txt](https://packetai.gitbook.io/packetai-autonomous-it-monitoring/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://packetai.gitbook.io/packetai-autonomous-it-monitoring/set-up/using-logstash-agent.md).

# Using Logstash agent

This document describes how to send logs to [PacketAI](https://packetai.co) using the logstash http output plugin

### Setting keys, cluster name and access control

In order to complete the setup, there are 2 main things to configure : 

1. Obtain your keys YOUR\_PAI\_TOKEN and  YOUR\_PAI\_IID by going to [**https://logpatterns.packetai.co/deploy/agent**](https://logpatterns.packetai.co/deploy/agent)**.** Pick any of the integrations to find your keys.
2. Replace `yourclustername` with your cluster name in the mutate block (see [below](#filter-conf)). :warning:Note that even if you are not using a proper cluster, you must provide a value for this variable. 

### Input conf

Logstash supports several input plugins, the list of input plugins it supports can be found [here](https://www.elastic.co/guide/en/logstash/current/input-plugins.html). The following file input configuration is provided as an example.

```
input { 
    file {
        path => [
            "/var/log/*.log",
            "/var/log/*/*.log"
        ]
    }
}
```

### Filter conf

Logstash supports several plugins for filtering logs, the more details about several filters (plugins) available [here](https://www.elastic.co/guide/en/logstash/current/filter-plugins.html).  PacketAI needs some additional fields needs to be attached to each message, see the below mutate section. We need to map the each application message to the following field, `[kubernetes][controller][name]` . for example if we have field `application_name` in the message, we could map `[kubernetes][controller][name] => [application_name]`. The following fields can be static

```
mutate {
    add_field => {
        "[kubernetes][namespace]" => "user" #don't change this value
        "[kubernetes][controller][type]" => "Deployment" #don't change this value
        "[kubernetes][controller][name]" => "test" #this needs to be mapped dynamically, it could be your application name
        "[packetai][cluster_name]" => "yourclustername" #needs to be update by user
    }
}
```

### Output conf

Logstash several output plugins, more details can be found [here](https://www.elastic.co/guide/en/logstash/current/output-plugins.html). PacketAI use http output plugin of logstash to send the messages to packetai. The http output plugin code snippet can be found below. we need to replace the YOUR\_PAI\_TOKEN\_HERE and YOUR\_PAI\_IID\_HERE with the correct values.

```
output {
  http {
    id => "packetai_logstash_ingester"
    url => "https://vector-ingester-logpatterns.packetai.co/vector/log"
    http_method => "post"
    headers => ["X-PAI-TOKEN","YOUR_PAI_TOKEN_HERE","X-PAI-IID","YOUR_PAI_IID_HERE"]
    format => "json_batch"
    http_compression => true
  }
}

```

### Sample logstash configuration

Please replace the YOUR\_PAI\_TOKEN\_HERE and YOUR\_PAI\_IID\_HERE with the correct values from the PacketAI account.

<pre><code>input { 
    file {
        path => [
            "/var/log/*.log",
            "/var/log/*/*.log"
        ]
    }
}
filter {
    ruby {
        #Here We are setting logfilename as app_name
        code => 'event.set("app_name",event.get("[log][file][path]").split("/")[-1][0..-5].gsub(/[^0-9a-zA-Z]/i, ""))'
    }
<strong>    mutate {
</strong>        add_field => {
            "[kubernetes][namespace]" => "user"
            "[kubernetes][controller][type]" => "Deployment"
            "[kubernetes][controller][name]" => "%{app_name}" # here we are setting the app_name as kubernetes.controller.name
            "[packetai][cluster_name]" => "prod"
        }
    }
}
output {
  http {
    id => "packetai_logstash_ingester"
    url => "https://vector-ingester-logpatterns.packetai.co/vector/log"
    http_method => "post"
    headers => ["X-PAI-TOKEN","YOUR_PAI_TOKEN_HERE","X-PAI-IID","YOUR_PAI_IID_HERE"]
    format => "json_batch"
    http_compression => true
  }
}

</code></pre>

### Validate your configuration

It is advised to test and validate your logstash configuration with below command

```
logstash --config.test_and_exit -f logstash-simple.conf
```