Quiver Transform table

Questions
1

I need to perform numerous transformations on an object set. I’m aware that there’s a limit of 50,000 objects (rows) per operation. If I attempt to transform the entire set of 112,000 objects in one go, it will be limited to the first 50,000. How can I make transformations on the whole set? Do I need to divide the whole Object set into smaller batches and apply the transformations iteratively. Any thought will be helpful.

Question originally asked by Lex Knape on Stack Overflow: palantir foundry - Quiver Transform table - Stack Overflow

2

For future readers’ reference, the screenshot below is what @lex-knape is referring to (all data is notional).

screenshot of 50k limit for transforms in Quiver
screenshot of 50k limit for transforms in Quiver2880×1600 457 KB

You can indeed split the object set into smaller sets and then perform your transforms. The result will be several transform tables, which you can then union into a single transform table.

An alternative is to use Contour to do this analysis on the dataset backing the object type (or the materialized edits dataset if this object type has received edits).

Answer originally provided by Ontologize on Stack Overflow: palantir foundry - Quiver Transform table - Stack Overflow