Column names in the Object

Questions
1

I used the Object set materialization card to use more than 50K rows and use the information in a dashboard. Can I change the name of the columns in the Object set materialisation card and if so can somebody explain how?

2

Hello! You can do this by adding in a transform table that takes the materialized object set as an input. Specifically, you’ll use the “rename columns” transform. In the configuration on the right (click on the cog if you don’t see it) you can change the name. Note in the screenshot how you have to click on the right side, rather than the left, in order to change the name.

annotated sc for renaming quiver transform table
annotated sc for renaming quiver transform table3284Ă—1850 356 KB

3

Hey Gena, thanks for your response. I managed to figure it out, but I have a question regarding the 50K rows limit. It seems counterintuitive for Foundry, a big data tool, to impose such a limit. To work around this, I utilized materialization to exceed the 50K limit and then used the transform table to rename columns. However, I’m unsure if using the transform table on materialized data still imposes the 50K limit on my object set. If so, can I rename columns in materialization and work with more than 50K rows? Thanks in advance.

4

@Palmakers if you’re working with materializations, then you’re free of the 50k limit, including if you use a transformation table on the materialization (e.g. to edit the column names)

1 Like
5

Thanks Taylor, it is sometimes confusing.

6

Oops – I was incorrect above. You still hit the limitation with a Transformation table. You can avoid it by using an Expression on the Materialization.

7

Taylor, thanks for your response. We’re facing a significant challenge with a dataset of 135 million objects, requiring numerous transformations like filtering and adding new columns. We’re contemplating the best approach:

  1. Performing transformations in the data lineage with PySpark, although this may result in loss of some pure data along the way.
  2. Using Quiver, although materialization doesn’t allow for column name changes.
  3. Leveraging TypeScript functions and integrating them into Workshop.

Any insights or recommendations on the most effective method would be greatly appreciated.