Log Management Collection Sources

A log source is a software or hardware component that produces log data. Multiple types of sources exist, and multiple methods exist to retrieve log data from the sources. The Alert Logic console allows you to create, edit, and update log collection sources, archive or restore old sources, and perform other tasks. You must create a collection policy before you can create a collection source. You can only create one collection source per host.

Log Manager supports the following log collection types:

All deployments:

• Flat file logs—A collection of text-based files from the host file system
• Syslog—A way for network devices to send event messages to a logging server
• Windows event logs—A Windows log file that tracks significant events, like user logins or program errors, on a Windows server

AWS deployments only:

• AWS CloudTrail logs—Log files that record AWS API calls for your account
• AWS S3 logs—Access log records that provide details about a single access request, such as the requester, bucket name, request time, request action, response status, and any error codes

Azure deployments only:

• Azure Activity logs—Logs that provide insight into the operations performed on resources in your subscription
• Azure App Service web server logs—Logs that provide detailed error information for HTTP failure status codes, failed requests, or HTTP transactions using the W3C extended log file format
• Azure SQL auditing logs—Logs that provide information on database events

After you provision and install the Alert Logic agent, the agent configures a default collection source for each log host in your system. You must create and configure new collection sources with existing collection policies to meet more specific requirements.

Access the Log Manager Log Sources page

To access the Log Manager hosts and sources page, click CONFIGURATION, and then click Deployments. From the Deployments page, click the Manual Deployments tile, and then click Log Sources.

Flat file logs

Create flat file source

Before you can create a flat file collection source, you must create a flat file collection policy. For more information, see Create a flat file policy.

To create a flat file collection source:

1. Access the Log Manager Log Sources page.
2. Click Log Sources.
3. Click the Add icon ().
4. From Source Log Type, select Flat File Collection.
5. In Source Name, enter a descriptive name.
6. Activate the Enable Collection switch if not activated.
7. Select Use an Appliance or Use an Agent, as appropriate to your setup.
   • To collect via an appliance, select a Collector and the corresponding IP address.
   • To collect via an agent, select Use an Agent and Select a Host from the drop-down menu.
8. Select Use an existing Policy or Create a New Policy.
   • For an existing policy, select the policy from the drop-down menu.
   • For creating a new policy, fill in all the necessary data.
9. Under Collection Alerts, select one or multiple alert options.
10. Select the correct Time Zone.
11. In Tags, type an easily filtered tag.
12. Click SAVE.

Update a flat file collection source

To update a flat file collection source:

1. From the Deployments page, click the deployment for which you want to update the log source.
2. In the left navigation area, click Log Sources.
3. Place your cursor over the desired collection source and click the pencil icon ().
4. Make the necessary updates.
5. Click SAVE.

Syslog logs

Log Manager accepts syslog files without additional configuration.

Windows event logs

Create a Windows event log collection source

You must create a Windows event log collection policy before you can create a Windows event log collection source.

For more information, see Create a Windows event log collection policy.

To create a Windows event log collection source:

1. Access the Log Manager Log Sources page.
2. Click the Add icon ().
3. From Source Log Type, select Windows Event Log.
4. In the Source Name field, type a descriptive name.
5. In Enable Collection switch, keep the default Enabled selection (to the right).
6. In Collection Method, select a Collector, and enter the IP Address.
7. In Collection Policy:
   • To use an existing policy, select Use an existing Policy and select a policy.
   • To create a new policy, select Create New Policy and select the settings you want. See Create a Windows event log collection policy for more details.
8. In Collection Alerts, click the field and select one or multiple alert options.
9. From Time Zone, select a time zone.
10. In the Tags field, type an easily filtered tag.
11. Click SAVE.

Update a Windows event log collection source

To update a Windows event log collection source:

1. From the Deployments page, click the deployment for which you want to update the log source.
2. In the left navigation area, click Log Sources.
3. Place your cursor over the desired collection source and click the pencil icon ().
4. Make the necessary updates.
5. Click SAVE.

AWS CloudTrail logs

Create an AWS CloudTrail collection source

You must create an AWS CloudTrail log source in the Alert Logic console to collect CloudTrail logs. To complete this action, you need the following AWS account information:

• SQS queue name
• IAM role credentials

To create an AWS CloudTrail collection source:

1. In the Alert Logic console, click CONFIGURATION, and then click Deployments.
2. Click the deployment tile for which you want to create a CloudTrail collection source.
3. In the left navigation area, click Log Sources.
4. Click the Add icon ().
5. From Source Log Type, select AWS CloudTrail.
6. In Source Name, type a descriptive name.
7. For the Enable Collection switch, keep the default Enabled selection (to the right).
8. In Collection Alerts, click the field and select one or more alert options.
9. In the SQS Queue Name field, type the name of the SQS queue you created in the previous steps.
10. From AWS Region, specify the region in which you created the SQS queue in the previous steps.
11. Select or create a new IAM Role.
• To use an existing IAM Role, select Use an existing IAM Role, and then select the IAM Role to use.
• To create a new IAM Role, select Create a new IAM Role, and then complete the following fields:
  • In Credential Name, enter a descriptive name.
  • In Role ID, enter the Role ARN you previously copied.
  • In External ID, enter the external ID you previously used.
12. Click Save.

Update an AWS CloudTrail collection source

1. From the Deployments page, click the deployment for which you want to update the log source.
2. In the left navigation area, click Log Sources.
3. Place your cursor over the desired collection source and click the pencil icon ().
4. Make the necessary updates.
5. Click SAVE.

AWS S3 logs

Create an AWS S3 collection source

To create an AWS S3 collection source:

1. In the Alert Logic console, click CONFIGURATION, and then click Deployments.
2. Click the Manual Deployments tile.
3. In the left navigation area, click Hosts and Sources.
4. Click the Sources tab.
5. Click the Add icon ().
6. From Source Log Type, select S3.
7. In Source Name, type a descriptive name.
8. Keep the Enable Collection switch set to Enabled (to the right).
9. In Bucket, type the bucket name, followed by the directory name. This bucket name must use a DNS-compliant name. For more information, visit the AWS documentation site.


s3bucketname/root_folder

10. In File Name or Pattern, type the file name or date pattern of the file log.
11. In Collection Policy:
    • To use an existing policy, select Use an existing Policy, and then select a policy.
    • To create a new policy, select Create New Policy and select the settings you want. For more information, see Create a S3 collection policy.
12. In Collection Alerts, click the field and select one or more alert options.
13. From Time Zone, select a time zone.
14. Select or create a new IAM Role.
    • To use an existing IAM Role, select Use an existing IAM Role. Next, in Existing IAM Role, select the IAM Role to use.
    • To create a new IAM Role, select Create a new IAM Role, and then complete the missing fields:
      • In Credential Name, enter a descriptive name.
      • In Role ID, enter the Role ARN you previously copied.
      • In External ID, enter the external ID you previously used.
15. In Collection Internal, type a value, in minutes, to indicate how often Log Manager retrieves S3 logs.
16. In the Tags field, type an easily filtered tag.
17. Click Save.

Update a S3 collection source

1. From the Deployments page, click the deployment for which you want to update the log source.
2. In the left navigation area, click Log Sources.
3. Place your cursor over the desired collection source and click the pencil icon ().
4. Make the necessary updates.
5. Click SAVE.

Azure audit logs

Create an Azure audit logs collection source

To create an Azure audit logs collection source:

Step 1 Name your new collection source

1. Access the Log Manager Hosts and Sources page.
2. Click Sources.
3. Click the Add icon ().
4. From Source Log Type, select Azure Activity Logs.
5. In the Source Name field, type a descriptive name.
6. In Enable Collection, keep the default selection Enabled. If not enabled, Log Manager will not collect from this source.
7. Select one of the following:
   • To use an existing Audit Account, select Existing Audit Account and select the Azureaccount you want to use.
   • To create a new Audit Account, select Add new Audit Account and select the settings you want. You will be asked to create a new user name and password.

   If you are adding a new Azure Audit Account to the Alert Logic system, verify it has the proper permissions allowing Alert Logic to read the Azure Audit events.

   To properly set up a role with the minimum permissions required, you’ll need to create a custom role in Azure RBAC, read Create custom roles for Azure Role-Based Access Control.

   The role below provides a minimum set of permissions required for Audit Log collection:

   { 
        "Name":  "<name of your role>",
        "Id": "<auto-assigned>",
        "IsCustom":  true,
        "Description":  "<description of the role>",
        "Actions":       [
                             "Microsoft.Authorization/*/read",
                             "Microsoft.Insights/eventtypes/*/read"
                         ],
        "NotActions":    [
   
                         ],
        "AssignableScopes":   [
                                  "/subscriptions/<add your Subscription ID>"
                              ]
   }
   

8. In Collection Alerts, click the field and select one or multiple alert options.
9. In Subscription ID, type your Azure Subscription ID.
10. In Resource Group Filter, type a Resource Group name.
11. In the Tags field, type an easily filtered tag.
12. Click Save.

Update an Azure Activity Logs collection source:

1. From the Deployments page, click the deployment for which you want to update the log source.
2. In the left navigation area, click Log Sources.
3. Place your cursor over the desired collection source and click the pencil icon ().
4. Make the necessary updates.
5. Click SAVE.

Azure App Service web server logs

Create an Azure App Service web server logs collection source

To create an Azure App Service web server logs collection source:

1. Access the Log Manager Log Sources page.
2. Click the Add icon ().
3. From Source Log Type, select App Service Web Server Logging.
4. In the Source Name field, type a descriptive name.
5. In Enable Collection, keep the default selection Enabled.
6. Select one of the following:
   • To use an existing Storage Account, select Existing Storage Account and select the Storage Account you want to use.
   • To create a new Storage Account, select Add new Storage Account and select the settings you want. You will be asked to create a new user name and password.

   In the Azure Portal, navigate to your storage account in which you store your logs, click Settings, and then click Access keys to view, copy, and regenerate your account access keys.

7. In Collection Alerts, click the field and select one or multiple alert options.
8. In App Service Name, type the name of your App Service Web application.
9. In Storage Blob Container, type your Storage Account Container name where your Web server logging is located.
10. In the Tags field, type an easily filtered tag.
11. Click Save.

Update an Azure App Service web server logs collection source:

1. From the Deployments page, click the deployment for which you want to update the log source.
2. In the left navigation area, click Log Sources.
3. Place your cursor over the desired collection source and click the pencil icon ().
4. Make the necessary updates.
5. Click SAVE.

Azure SQL auditing logs

Create an Azure SQL database audit logs collection source

To create an Azure SQL database audit logs collection source:

1. Access the Log Manager Log Sources page.
2. Click the Add icon ().
3. From Source Log Type, select Azure SQL Auditing.
4. In the Source Name field, type a descriptive name.
5. In Enable Collection, keep the default selection Enabled.
6. Select one of the following:
   • To use an existing Storage Account, select Existing Storage Account and select the Storage Account you want to use.
   • To create a new Storage Account, select Add new Storage Account and select the settings you want. You will be asked to fill in your Credential Name, Storage Account Name, and Access Key.

   In the Azure Portal, navigate to your storage account where your logs are stored, click Settings, and then click Access keys to view, copy, and regenerate your account access keys.

7. In Collection Alerts, click the field and select one or multiple alert options.
8. In the Azure SQL Table Name, type your SQL Table Name between SQLDBAuditLogs and YYYYMMDD.
9. In the Tags field, type an easily filtered tag.
10. Click Save.

Update an Azure SQL database audit logs collection source:

1. From the Deployments page, click the deployment for which you want to update the log source.
2. In the left navigation area, click Log Sources.
3. Place your cursor over the desired collection source and click the pencil icon ().
4. Make the necessary updates.
5. Click SAVE.

Archive and restore collection sources

Archive a collection source to remove the collection source entry from the Alert Logic console.

Archive a collection source

To archive a collection source:

1. Access the Log Manager Hosts and Sources page.
2. Click Sources.
3. Place your cursor over the desired collection source and click the box icon ().
4. Click Archive.

Restore an archived collection source

To restore an archived log source:

1. Access the Log Manager Hosts and Sources page.
2. Click Sources.
3. Above the log source table, set the switch to show archived sources.
4. Place your cursor over the desired collection source and click the box icon ().
5. Click Restore.

Additional tasks

View collection source information

To view information about a collection source:

1. Access the Log Manager Hosts and Sources page.
2. Click Sources.
3. Place your cursor over the desired collection source and click on it. A tray will appear with three different tabs:
   • Details: This tab displays all information about the collection source, including the account number, the public host name, when it was created or modified, and the host ID.
   

   The Status field lists any current errors.

   • Metadata History: This tab displays only the metadata history for the collection source.
   • Status History: This tab displays only the status history, including the current status of the collection source.

Add a source to a case

To add a source to a case:

1. At the top of the Alert Logic console, click Log Manager, then Deployments.
2. Click the deployment tile you want to modify.
3. In the left navigation area, click Hosts and Sources.
4. Click the Sources tab.
5. Place your cursor over the desired source and click the suitcase icon ( ).
6. Click Add.

Related topics