Plugin Release Checklist

All plugins are currently hosted on the Tailpipe Hub. If you want to contribute one -- and we hope you do! -- here are the most common things we ask contributors to check to prepare for the plugin's release. Feel free to tick the boxes as you go through the list!

• Basic Configuration
• Configuration File
• Credentials
• Table and Column Names
• Table and Column Descriptions
• Table and Column Design
• Documentation
• Final Review

Basic Configuration

The repository name should use the format tailpipe-plugin-<pluginName>, e.g., tailpipe-plugin-aws, tailpipe-plugin-azure, tailpipe-plugin-github. The plugin name should be one word, so there are always 3 parts in the repository name.

To help with discoverability in GitHub, the repository topics should include:

• duckdb
• sql
• tailpipe
• tailpipe-plugin

The repository website/homepage should link to the Hub site. The URL is composed of the GitHub organization and plugin name, for instance:

• https://github.com/turbot/tailpipe-plugin-aws results in https://hub.tailpipe.io/plugins/turbot/aws

The Go version in go.mod and any workflows is 1.23.

The .goreleaser.yml file uses the standard format, e.g., AWS plugin .goreleaser.yml.

A CHANGELOG.md is included and contains release notes for the upcoming version (typically v0.1.0).

The plugin uses 2 licenses:

• Most of the plugin uses the Apache License 2.0
• Except for the docs which use the CC BY-NC-ND License 4.0

The Makefile file is present and builds to the correct plugin path.

Credentials

If there's a Terraform provider for your API, the plugin supports the same credential methods as the provider.

When there are commonly used CLI credentials, like .aws/credentials, the plugin works with them.

When credentials expire, and the API's SDK does not automatically refresh them, the plugin alerts the user and tells them how to refresh.

It's possible to set credentials using an environment variable if the API's SDK also supports using environment variables.

Table and Column Names

Table names use the format <plugin>_<log type>, e.g., aws_cloudrail_log, aws_s3_server_access_log, azure_activity_log, github_audit_log.

Column names should use the lower snake case form of the log property. For instance, in aws_cloudtrail_log, the userIdentity property becomes the user_identity column.

Table and Column Descriptions

Every table and column has a description. These are consistent across tables. The default tp_* column descriptions should be overridden if there's additional log specific information in the respective tp_* column.

Table and Column Design

The table enriches the row with the following required common fields:

• tp_date - The date the event was originally generated.
• tp_id - A unique identifier for the row. In Turbot plugins, typically set to an xid.
• tp_index - The index used to partition the data, e.g., AWS account ID, GitHub organization, hostname.
• tp_ingest_timestamp - The timestamp when the event was ingested into the system.
• tp_timestamp - The timestamp when the event was originally generated.

Additional common fields, like tp_ips and tp_source_location should be added based on what is available in the log type.

For each supported artifact source, the table defines a default FileLayout that uses the most common pattern. For instance, the aws_cloudtrail_log uses the standard AWS log format as the default FileLayout for the aws_s3_bucket source as part of its GetSourceMetadata function.

Logging

When the plugin returns an error, it includes the location and any related args, along with the error itself.

Column Types

Struct columns are preferred, so columns with nested properties should use the STRUCT type if the property structure is known; however, if a column has unknown properties, it should use the JSON type.

Documentation

Index Documentation

The index document contains a front matter block, like the one below:

The category is an appropriate choice from the list at hub.tailpipe.io/plugins.

The icon URL is a link to an .svg file hosted on hub.tailpipe.io. Please request an icon through the Turbot Community Slack and a URL will be provided to use in this variable.

The color matches the provider's brand guidelines, typically stated on a page like this one for Amazon Alexa.

The description in docs/index.md is appropriate for the provider. The AWS plugin, for example, uses:

The opening sentence of the Wikipedia page for the provider can be a good source of guidance here.

If a plugin has at least one table with a matching Detections mod, e.g., AWS CloudTrail Log Detections, then the Detections as Code with Powerpipe section is added and links to the mod.

If a plugin defines a connection type, the plugin should:

• List connection arguments
• Link to provider documentation
• Explain how to use existing CLI creds when that's possible

Table Documentation

Each table document shows a basic example of how to configure a partition with a connection (if applicable) and then collect log data from that partition.

Each table document shows 3-5 useful examples that reflect real-world scenarios.

All examples should specify columns and should usually include the log's timestamp and actor (if available) as the first column. Using SELECT * is generally not preferred as it can produce too much data to be helpful.

Source Documentation

Each source document shows 3-5 configuration examples that users can copy, modify, and use to collect logs easily.

All source arguments should be listed along with their types and descriptions, and if they're required.

For each table that defines defaults for the source's arguments, a link should be added to the relevant table's Source Defaults section. For instance, the aws_s3_bucket document links to aws_cloudtrail_log - Source Defaults: aws_s3_bucket.

Final Review

The plugin has been tested with real log data.

The example in README.md matches the one in docs/index.md.

Each plugin has an icon shown on the Hub. Please request an icon through the Turbot Community Slack.

The plugin really nails easy setup, there's a short path to a first successful query, and it runs quickly.

You've considered, and addressed, reasons why this plugin could fail to delight its community.