FAQ
Why does collection report fewer rows than expected?
Tailpipe maintains collection state, so you'll only see new rows.
How do I reset a table?
If you want to reset and collect everything, you can use rm -rf ~/.tailpipe/internal/collection/default/*. You might want to do this if you're building your own plugin and changing the schema.
NoteWhat's the story for published plugins that change schema?
Can I use DuckDB to query my Tailpipe tables?
Yes. Use tailpipe connect to acquire a unique connection string. For example:
Then connect to it:
Can I define more than one partition per table?
Yes. In this example, the aws_cloudtrail_log table has two partitions, one for each of two AWS accounts.
Can I define multiple sources for a table?
Yes. In this example, the aws_cloudtrail_log table has one partition that encompasses two AWS accounts.
What partition indexes are available for a table?
That depends on how the plugin author has defined the common tp_index field. For AWS tables, it's the account_id. In the dual-partition case above, you could carve the logs by account_id using the common tp_partition field (but tp_index will always be the same). In the single-partition case above, you could carve the logs by account_id using tp_index (but tp_partition will aways be the same).