# Using Logstash agent

This document describes how to send logs to [PacketAI](https://packetai.co) using the logstash http output plugin

### Setting keys, cluster name and access control

In order to complete the setup, there are 2 main things to configure : 

1. Obtain your keys YOUR\_PAI\_TOKEN and  YOUR\_PAI\_IID by going to [**https://logpatterns.packetai.co/deploy/agent**](https://logpatterns.packetai.co/deploy/agent)**.** Pick any of the integrations to find your keys.
2. Replace `yourclustername` with your cluster name in the mutate block (see [below](#filter-conf)). :warning:Note that even if you are not using a proper cluster, you must provide a value for this variable. 

### Input conf

Logstash supports several input plugins, the list of input plugins it supports can be found [here](https://www.elastic.co/guide/en/logstash/current/input-plugins.html). The following file input configuration is provided as an example.

```
input { 
    file {
        path => [
            "/var/log/*.log",
            "/var/log/*/*.log"
        ]
    }
}
```

### Filter conf

Logstash supports several plugins for filtering logs, the more details about several filters (plugins) available [here](https://www.elastic.co/guide/en/logstash/current/filter-plugins.html).  PacketAI needs some additional fields needs to be attached to each message, see the below mutate section. We need to map the each application message to the following field, `[kubernetes][controller][name]` . for example if we have field `application_name` in the message, we could map `[kubernetes][controller][name] => [application_name]`. The following fields can be static

```
mutate {
    add_field => {
        "[kubernetes][namespace]" => "user" #don't change this value
        "[kubernetes][controller][type]" => "Deployment" #don't change this value
        "[kubernetes][controller][name]" => "test" #this needs to be mapped dynamically, it could be your application name
        "[packetai][cluster_name]" => "yourclustername" #needs to be update by user
    }
}
```

### Output conf

Logstash several output plugins, more details can be found [here](https://www.elastic.co/guide/en/logstash/current/output-plugins.html). PacketAI use http output plugin of logstash to send the messages to packetai. The http output plugin code snippet can be found below. we need to replace the YOUR\_PAI\_TOKEN\_HERE and YOUR\_PAI\_IID\_HERE with the correct values.

```
output {
  http {
    id => "packetai_logstash_ingester"
    url => "https://vector-ingester-logpatterns.packetai.co/vector/log"
    http_method => "post"
    headers => ["X-PAI-TOKEN","YOUR_PAI_TOKEN_HERE","X-PAI-IID","YOUR_PAI_IID_HERE"]
    format => "json_batch"
    http_compression => true
  }
}

```

### Sample logstash configuration

Please replace the YOUR\_PAI\_TOKEN\_HERE and YOUR\_PAI\_IID\_HERE with the correct values from the PacketAI account.

<pre><code>input { 
    file {
        path => [
            "/var/log/*.log",
            "/var/log/*/*.log"
        ]
    }
}
filter {
    ruby {
        #Here We are setting logfilename as app_name
        code => 'event.set("app_name",event.get("[log][file][path]").split("/")[-1][0..-5].gsub(/[^0-9a-zA-Z]/i, ""))'
    }
<strong>    mutate {
</strong>        add_field => {
            "[kubernetes][namespace]" => "user"
            "[kubernetes][controller][type]" => "Deployment"
            "[kubernetes][controller][name]" => "%{app_name}" # here we are setting the app_name as kubernetes.controller.name
            "[packetai][cluster_name]" => "prod"
        }
    }
}
output {
  http {
    id => "packetai_logstash_ingester"
    url => "https://vector-ingester-logpatterns.packetai.co/vector/log"
    http_method => "post"
    headers => ["X-PAI-TOKEN","YOUR_PAI_TOKEN_HERE","X-PAI-IID","YOUR_PAI_IID_HERE"]
    format => "json_batch"
    http_compression => true
  }
}

</code></pre>

### Validate your configuration

It is advised to test and validate your logstash configuration with below command

```
logstash --config.test_and_exit -f logstash-simple.conf
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://packetai.gitbook.io/packetai-autonomous-it-monitoring/set-up/using-logstash-agent.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.