Deployment Assets: Log Sources

A log source is a software or hardware component that produces log data. Multiple types of sources exist, and multiple methods exist to retrieve log data from the sources. The Alert Logic console allows you to create, edit, and update log collection sources, archive or restore old sources, and perform other tasks.

Alert Logic supports the following log collection types:

All deployments:

  • Flat file logs—A collection of text-based files from the host file system
  • Syslog—A way for network devices to send event messages to a logging server
  • Windows event logs—A Windows log file that tracks significant events, like user logins or program errors, on a Windows server

AWS deployments only:

  • AWS CloudTrail logs—Log files that record AWS API calls for your account
  • AWS S3 logs—Access log records that provide details about a single access request, such as the requester, bucket name, request time, request action, response status, and any error codes

Azure deployments only:

  • Azure Activity logs—Logs that provide insight into the operations performed on resources in your subscription
  • Azure App Service web server logs—Logs that provide detailed error information for HTTP failure status codes, failed requests, or HTTP transactions using the W3C extended log file format

After you provision and install the Alert Logic agent, the agent configures a default collection source for each log host in your system. You must create and configure new collection sources with existing collection policies to meet more specific requirements. For more information about Log Management policies, see Log Manager policies.

Create and maintain flat file log collection sources

Before you can create a flat file collection source, you must create a flat file collection policy. For more information, see Create a flat file policy.

To create a flat file collection source:

  1. From the Deployments page, click the deployment for which you want to create a flat file collection source.
  2. Click Log Sources.
  3. Click the Add icon ( ).
  4. From Source Log Type, select Flat File Collection.
  5. In Source Name, enter a descriptive name.
  6. Select Enable Collection.
  7. Select Use an Appliance or Use an Agent, as appropriate to your setup.
    • To use an appliance to collect flat file logs, select a Collector and the corresponding IP address.
    • To use an agent to collect flat file logs, select Use an Agent, and then Select a Host from the drop-down menu.
  8. Select Use an existing Policy or Create a New Policy.
  9. Under Collection Alerts, select one or more alert options.
  10. Select the correct Time Zone.
  11. In the Tags field, type an easily filtered tag.
  12. Click SAVE.

To update a flat file collection source:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. In the left navigation area, click Log Sources.
  3. Place your cursor over the desired collection source and click the pencil icon ().
  4. Make the necessary updates.
  5. Click SAVE.

Syslog collection

Log Management collects syslog files without additional configuration.

Create and maintain Windows event log collection sources

Before you can create a Windows event log collection source, you must create a Windows event log collection policy. For more information, see Create a Windows event log collection policy.

To create a Windows event log collection source:

  1. From the Deployments page, click the deployment for which you want to create a Windows event log collection source.
  2. Click Log Sources.
  3. Click the Add icon ().
  4. From Source Log Type, select Windows Event Log.
  5. In the Source Name field, type a descriptive name.
  6. Select Enable Collection.
  7. Under Collection Method, select a Collector, and type the IP Address.
  8. Determine whether you want to use an existing policy or create a new policy.
    • To use an existing policy, select Use an existing Policy, and then select a policy.
    • To create a new policy, select Create New Policy, and provide the necessary information. For more information, see Create a Windows event log collection policy.
  9. Under Collection Alerts, select one or more alert options.
  10. Select a Time Zone.
  11. In the Tags field, type an easily filtered tag.
  12. Click SAVE.

To update a Windows event log collection source:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. In the left navigation area, click Log Sources.
  3. Place your cursor over the desired collection source and click the pencil icon ().
  4. Make the necessary updates.
  5. Click SAVE.

Create and maintain AWS CloudTrail log collection sources

Though this feature appears to all users, only those with an AWS account can utilize it.

You must create an AWS CloudTrail log source in the Alert Logic console to collect CloudTrail logs. To complete this action, you need the following AWS account information:

  • SQS queue name
  • IAM role credentials

To create an AWS CloudTrail collection source:

  1. From the Deployments page, click the deployment for which you want to create a CloudTrail log collection source.
  2. In the left navigation area, click Log Sources.
  3. Click the Add icon ().
  4. From Source Log Type, select AWS CloudTrail.
  5. In Source Name, type a descriptive name.
  6. Select Enable Collection.
  7. In Collection Alerts, click the field and select one or more alert options.
  8. In the SQS Queue Name field, type the name of the SQS queue you created to collect CloudTrail logs.
  9. From AWS Region, specify the region in which you created the SQS queue in the previous steps.
  10. Select or create a new IAM Role.
    • To use an existing IAM Role, select Use an existing IAM Role, and then select the IAM Role to use.
    • To create a new IAM Role, select Create a new IAM Role, and then complete the following fields:
      • In Credential Name, enter a descriptive name.
      • In Role ID, enter the Role ARN you previously copied.
      • In External ID, enter the external ID you previously used.
  11. Click SAVE.

To update an AWS CloudTrail collection source:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. In the left navigation area, click Log Sources.
  3. Place your cursor over the desired collection source and click the pencil icon ().
  4. Make the necessary updates.
  5. Click SAVE.

Create and maintain AWS S3 log collection sources

Though this feature appears to all users, only those with an AWS account can utilize it.

To create an AWS S3 collection source:

  1. From the Deployments page, click the deployment for which you want to create an S3 collection source.
  2. In the left navigation area, click Log Sources.
  3. Click the Add icon ().
  4. From Source Log Type, select S3.
  5. In Source Name, type a descriptive name.
  6. Select Enable Collection.
  7. In Bucket, type the bucket name, followed by the directory name. This bucket name must use a DNS-compliant name. For more information, visit the AWS documentation site.

    8. s3bucketname/root_folder

  8. In File Name or Pattern, type the file name or date pattern of the file log.
  9. In Collection Policy:
    • To use an existing policy, select Use an existing Policy, and then select a policy.
    • To create a new policy, select Create New Policy and select the settings you want. For more information, see Create a S3 collection policy.
  10. In Collection Alerts, click the field and select one or more alert options.
  11. From Time Zone, select a time zone.
  12. Select or create a new IAM Role
    • To use an existing IAM Role, select Use an existing IAM Role. Next, in Existing IAM Role, select the IAM Role to use.
    • To create a new IAM Role, select Create a new IAM Role, and then complete the missing fields:
      • In Credential Name, enter a descriptive name.
      • In Role ID, enter the Role ARN you previously copied.
      • In External ID, enter the external ID you previously used.
  13. In Collection Internal, type a value, in minutes, to indicate how often Log Manager retrieves S3 logs.
  14. In the Tags field, type an easily filtered tag.
  15. Click SAVE.

To update an AWS S3 collection source:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. In the left navigation area, click Log Sources.
  3. Place your cursor over the desired collection source and click the pencil icon ().
  4. Make the necessary updates.
  5. Click SAVE.

Create and maintain Azure Audit log collection sources

Though this feature appears to all users, only those with an Azure account can utilize it.

To create an Azure Audit log collection source:

  1. From the Deployments page, click the deployment for which you want to create an audit log collection source.
  2. In the left navigation area, click Log Sources.
  3. Click the Add icon ().
  4. From Source Log Type, select Azure Audit Logs.
  5. In the Source Name field, type a descriptive name.
  6. Select Enable Collection.
  7. Select one of the following:
    • To use an existing audit account, select Existing Audit Account and select the Azure account you want to use.
    • To create a new audit account, select Add new Audit Account and select the settings you want. You will be asked to create a new user name and password.

    If you select Add new Audit Account, verify the account has the proper permissions to allow Alert Logic to read the Azure Audit events.

    To properly set up a role with the minimum permissions required, you must create a custom role in Azure. For more information, read Create custom roles for Azure Role-Based Access Control.

    The role below provides a minimum set of permissions required for Audit Log collection:

    { 
         "Name":  "<name of your role>",
         "Id": "<auto-assigned>",
         "IsCustom":  true,
         "Description":  "<description of the role>",
         "Actions":       [
                              "Microsoft.Authorization/*/read",
                              "Microsoft.Insights/eventtypes/*/read"
                          ],
         "NotActions":    [

                          ],
         "AssignableScopes":   [
                                   "/subscriptions/<add your Subscription ID>"
                           ]
}
  8. In Collection Alerts, select one or more alert options.
  9. In Subscription ID, type your Azure Subscription ID.
  10. In Resource Group Filter, type a Resource Group name.
  11. In the Tags field, type an easily filtered tag.
  12. Click SAVE.

To update Azure Audit logs collection sources:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. In the left navigation area, click Log Sources.
  3. Place your cursor over the desired collection source and click the pencil icon ().
  4. Make the necessary updates.
  5. Click SAVE.

Create and maintain Azure App Service web server logs

Though this feature appears to all users, only those with an Azure account can utilize it.

To create an Azure App Service web server logs collection source:

  1. From the Deployments page, click the deployment for which you want to create an Azure App ervice web server collection source.
  2. In the left navigation area, click Log Sources.
  3. Click the Add icon ().
  4. From Source Log Type, select App Service Web Server Logging.
  5. In the Source Name field, type a descriptive name.
  6. Select Enable Collection.
  7. Select one of the following:
    • To use an existing storage account, select Existing Storage Account and select the storage account you want to use.
    • To create a new storage account, select Add new Storage Account and select the settings you want. You will be asked to create a new user name and password.

    In the Azure Portal, navigate to the storage account in which you store your logs, click Settings, and then click Access keys to view, copy, and regenerate your account access keys.

  8. In Collection Alerts, click the field and select one or more alert options.
  9. In App Service Name, type the name of your App Service Web application.
  10. In Storage Blob Container, type the storage account container name where your web server logging is located.
  11. In the Tags field, type an easily filtered tag.
  12. Click SAVE.

To update an Azure App Service web server logs collection source:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. In the left navigation area, click Log Sources.
  3. Place your cursor over the desired collection source and click the pencil icon ().
  4. Make the necessary updates.
  5. Click SAVE.

Mass edit collection sources

The Mass Edit feature allows you to edit policies and tags for all sources, filtered sources, or sources you specify. Also, mass edit contains a mass archive feature.

To mass edit log sources:

  1. On the Log Sources page, click the gear icon ().
  2. Select Mass Edit.
  3. In Apply changes to, select:
    • All Sources to mass edit all sources
    • Only Filtered Sources to mass edit only filtered sources
    • Only Selected Sources to mass edit only selected sources
  4. From Collection Policy, select the collection policy to use.
  5. From Replace Collection Alerts, select an alert to apply to the selected sources.
This action overrides the current alerts that correspond to the selected sources. If you leave this option blank, current alerts will not change.
  1. Select Enable Collection.
  2. In Tags, select a tag option from the drop-down menu. Below, type a tag to follow the rule selected in the drop-down menu.
  3. In Archive Sources, select an option from the drop-down menu.
  4. Click SAVE.

Archive and restore log sources

Archive a collection source to remove the log source entry from the Log Sources page, and make it available for use at a later time.

Archive a collection source

To archive a collection source:

  1. From the Deployments page, click the deployment for which you want to archive log sources.
  2. Click Log Sources.
  3. Place your cursor over the desired collection source and click the Archive icon ().
  4. Click ARCHIVE.
You cannot archive a log host or collection source that stops log collection.

Restore an archived collection source

To restore an archived log source:

  1. From the Deployments page, click the deployment to which you want to restore an archived log source.
  2. Click Log Sources.
  3. Above the log source table, select Show Archive.
  4. Place your cursor over the desired collection source and click the Archive icon ().
  5. Click RESTORE.

Additional tasks

View collection source information

To view information about a collection source:

  1. Access the Log Manager Hosts and Sources page.
  2. Click Sources.
  3. Place your cursor over the desired collection source and click on it. A tray will appear with three different tabs:
    • Details: This tab displays all information about the collection source, including the account number, the public host name, when it was created or modified, and the host ID.

    The Status field lists any current errors.

    • Metadata History: This tab displays only the metadata history for the collection source.
    • Status History: This tab displays only the status history, including the current status of the collection source.