loki/tools/lambda-promtail

lambda-promtail

This is a sample template for lambda-promtail - Below is a brief explanation of what we have generated for you:

.
├── Makefile                    <-- Make to automate build
├── README.md                   <-- This instructions file
├── hello-world                 <-- Source code for a lambda function
│   └── main.go                 <-- Lambda function code
└── template.yaml

Requirements

• AWS CLI already configured with Administrator permission
• Docker installed
• Golang
• SAM CLI - Install the SAM CLI

Setup process

Installing dependencies & building the target

In this example we use the built-in sam build to automatically download all the dependencies and package our build target.
Read more about SAM Build here

The sam build command is wrapped inside of the Makefile. To execute this simply run

make

Local development

**Invoking function locally

make dry-run

Packaging and deployment

AWS Lambda Golang runtime requires a flat folder with the executable generated on build step. SAM will use CodeUri property to know where to look up for the application:

make build

To deploy your application for the first time, first make sure you've set the following parameters in the template:

• LogGroup
• PromtailAddress
• ReservedConcurrency

These can also be set via overrides by passing the following argument to sam deploy:

  --parameter-overrides           Optional. A string that contains AWS
                                  CloudFormation parameter overrides encoded
                                  as key=value pairs.For example, 'ParameterKe
                                  y=KeyPairName,ParameterValue=MyKey Parameter
                                  Key=InstanceType,ParameterValue=t1.micro' or
                                  KeyPairName=MyKey InstanceType=t1.micro

Also, if your deployment requires a VPC configuration, make sure to edit the VpcConfig field in the template.yaml manually.

Then run the following in your shell:

sam deploy --guided --capabilities CAPABILITY_IAM,CAPABILITY_NAMED_IAM --parameter-overrides PromtailAddress=<>,LogGroup=<>

The command will package and deploy your application to AWS, with a series of prompts:

• Stack Name: The name of the stack to deploy to CloudFormation. This should be unique to your account and region, and a good starting point would be something matching your project name.
• AWS Region: The AWS region you want to deploy your app to.
• Confirm changes before deploy: If set to yes, any change sets will be shown to you before execution for manual review. If set to no, the AWS SAM CLI will automatically deploy application changes.
• Allow SAM CLI IAM role creation: Many AWS SAM templates, including this example, create AWS IAM roles required for the AWS Lambda function(s) included to access AWS services. By default, these are scoped down to minimum required permissions. To deploy an AWS CloudFormation stack which creates or modified IAM roles, the CAPABILITY_IAM value for capabilities must be provided. If permission isn't provided through this prompt, to deploy this example you must explicitly pass --capabilities CAPABILITY_IAM to the sam deploy command.
• Save arguments to samconfig.toml: If set to yes, your choices will be saved to a configuration file inside the project, so that in the future you can just re-run sam deploy without parameters to deploy changes to your application.

Appendix

Golang installation

Please ensure Go 1.x (where 'x' is the latest version) is installed as per the instructions on the official golang website: https://golang.org/doc/install

A quickstart way would be to use Homebrew, chocolatey or your linux package manager.

Homebrew (Mac)

Issue the following command from the terminal:

brew install golang

If it's already installed, run the following command to ensure it's the latest version:

brew update
brew upgrade golang

Chocolatey (Windows)

Issue the following command from the powershell:

choco install golang

If it's already installed, run the following command to ensure it's the latest version:

choco upgrade golang

Limitations

• Error handling: If promtail is unresponsive, lambda-promtail will drop logs after retry_count, which defaults to 2.
• AWS does not support passing log lines over 256kb to lambdas.