In this guide we will configure a Lambda function which will be triggered every time new CloudWatch logs are generated, and which will forward the logs to Loom.
If you haven’t set up your CloudWatch log groups yet, follow these instructions.
In Loom, go to the Settings -> Add & Manage Data -> Data Inputs screen, and press the “Plug a new data input” button:
From the modal that opens, select the “AWS CloudWatch Logs” option.
First step - creating the Lambda function
The first step in the modal contains a link to the code of the function in a zip file, and a configuration file. Download both files, and add the configuration file to the zip.
Click the Next button, to move to the Create the Lambda function screen in the wizard.
- Browse to the AWS Management Console, click "Services", then "Lambda" (or simply click here)
- Click "Create function":
On the "Author from scratch" step:
- Give the function a name (e.g. loomCloudWatchLambda)
- Set the runtime to Python 2.7
- In the "Role" drop-down: Choose "Create a custom role".
A new tab will be opened. Click "Allow". The tab will close. Note: you must have the permission to create and assing roles.
- In the "Existing role" field, choose "lambda_basic_execution" Note: it might take a few minutes until the role is created and is available for selection.
- Click "Next".
On the next configuration step:
- under "Function code", choose "Upload a .Zip file" as "Code entry type" and upload the zip file previously created
- Change "Handler" to loom.handler.
- Under "Basic settings", set memory to 128 MB and timeout to 2 minutes (although the script usually executes much faster)
- Click "Save"
Second step - configure the CloudWatch trigger
NOTE: if your CloudWatch logs are already configured to trigger another Lambda, you'll have to follow the steps outlined here.
In the AWS Management Console, click "Services", then "CloudWatch" (or simply click here). On the left of the screen, select "Logs":
Select the log groups you'd like to stream.
Pay attention, not to select the log group of the Lambda functions, but of the instances you’d like to monitor.
Under "Actions", select "Stream to AWS Lambda":
Choose the Lambda you created in the previous steps, and click "Next":
On the "Configure Log Format and Filters" step:
- Choose "Other" as "Log Format"
- Optionally, apply filters to the events (documentation)
- Click "Next"
Click "Start Streaming"
That’s it. , You will see the new lambda under the subscription column:
Third step - viewing the data in Loom
Go back to Loom and close the new-data-input wizard. In the data-inputs screen, start the default-rest REST endpoint data input (if not already running).
after a few minutes you should start seeing events coming, you can press the examples button (eye) for viewing a sample of the incoming events.
That's it! If you have any questions or suggestions please let us know :)