Like Prometheus, but for logs.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
loki/tools/lambda-promtail/README.md

120 lines
4.7 KiB

# lambda-promtail
This is a sample template for lambda-promtail - Below is a brief explanation of what we have generated for you:
```bash
.
├── Makefile <-- Make to automate build
├── README.md <-- This instructions file
├── hello-world <-- Source code for a lambda function
│ └── main.go <-- Lambda function code
└── template.yaml
```
## Requirements
* AWS CLI already configured with Administrator permission
* [Docker installed](https://www.docker.com/community-edition)
* [Golang](https://golang.org)
* SAM CLI - [Install the SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)
## Setup process
### Installing dependencies & building the target
In this example we use the built-in `sam build` to automatically download all the dependencies and package our build target.
Read more about [SAM Build here](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-build.html)
The `sam build` command is wrapped inside of the `Makefile`. To execute this simply run
```shell
make
```
### Local development
**Invoking function locally
```bash
make dry-run
```
## Packaging and deployment
AWS Lambda Golang runtime requires a flat folder with the executable generated on build step. SAM will use `CodeUri` property to know where to look up for the application:
```bash
make build
```
To deploy your application for the first time, first make sure you've set the following parameters in the template:
- `LogGroup`
- `PromtailAddress`
- `ReservedConcurrency`
These can also be set via overrides by passing the following argument to `sam deploy`:
```
--parameter-overrides Optional. A string that contains AWS
CloudFormation parameter overrides encoded
as key=value pairs.For example, 'ParameterKe
y=KeyPairName,ParameterValue=MyKey Parameter
Key=InstanceType,ParameterValue=t1.micro' or
KeyPairName=MyKey InstanceType=t1.micro
```
Also, if your deployment requires a VPC configuration, make sure to edit the `VpcConfig` field in the `template.yaml` manually.
Then run the following in your shell:
```bash
sam deploy --guided --capabilities CAPABILITY_IAM,CAPABILITY_NAMED_IAM --parameter-overrides PromtailAddress=<>,LogGroup=<>
```
The command will package and deploy your application to AWS, with a series of prompts:
* **Stack Name**: The name of the stack to deploy to CloudFormation. This should be unique to your account and region, and a good starting point would be something matching your project name.
* **AWS Region**: The AWS region you want to deploy your app to.
* **Confirm changes before deploy**: If set to yes, any change sets will be shown to you before execution for manual review. If set to no, the AWS SAM CLI will automatically deploy application changes.
* **Allow SAM CLI IAM role creation**: Many AWS SAM templates, including this example, create AWS IAM roles required for the AWS Lambda function(s) included to access AWS services. By default, these are scoped down to minimum required permissions. To deploy an AWS CloudFormation stack which creates or modified IAM roles, the `CAPABILITY_IAM` value for `capabilities` must be provided. If permission isn't provided through this prompt, to deploy this example you must explicitly pass `--capabilities CAPABILITY_IAM` to the `sam deploy` command.
* **Save arguments to samconfig.toml**: If set to yes, your choices will be saved to a configuration file inside the project, so that in the future you can just re-run `sam deploy` without parameters to deploy changes to your application.
# Appendix
### Golang installation
Please ensure Go 1.x (where 'x' is the latest version) is installed as per the instructions on the official golang website: https://golang.org/doc/install
A quickstart way would be to use Homebrew, chocolatey or your linux package manager.
#### Homebrew (Mac)
Issue the following command from the terminal:
```shell
brew install golang
```
If it's already installed, run the following command to ensure it's the latest version:
```shell
brew update
brew upgrade golang
```
#### Chocolatey (Windows)
Issue the following command from the powershell:
```shell
choco install golang
```
If it's already installed, run the following command to ensure it's the latest version:
```shell
choco upgrade golang
```
## Limitations
- Error handling: If promtail is unresponsive, `lambda-promtail` will drop logs after `retry_count`, which defaults to 2.
- AWS does not support passing log lines over 256kb to lambdas.