Unlike data assets, which are brought into the platform from external sources, data products are datasets built from data assets already in the platform.
The ThinkData Platform is not only a data catalog for data assets that already exist across your organization, it is also a data refinery where you can create derived data products directly from the platform. Whether you are cleansing, transforming, migrating, or blending data, the platform UI and SDK can support end-to-end data productization.
A data product is a data table that has been derived from other data tables. It may be a filtered query of a single table, or a join between multiple tables. Data products can even help with data migration, since you can choose to replicate an entire data table into a new warehouse.
Unlike data assets, which are brought in from data sources, data products require a destination. Destinations are landing zones for data (such as a dedicated warehouse) that you can create and configure using the platform.
Before you get started: Make sure you have already set up a destination for your data product. You may also need the dataset connection name, dataset id and, if necessary, the keys of the column(s) included in the join (if performing).
Step 1: Creating a data product has the same opening step as creating a data asset. To begin, select the "+ Create Dataset" button from the Search or Data tab in the sidebar.
Step 2: Give your data product a name. You will not be able to save unless it has a title. This name can be changed later.
Step 3: In the dataset type section toggle from "Data asset" to "Data product". This will change the section from selecting a data source to pull data from to selecting a data destination to send data to. Select the destination where you would like the data product to be warehoused.
Step 4: Scroll to the bottom of the create dataset page and select "define data product". This will open a query builder for creating your data product.
Step 5: Selecting "define data product" will open the data transformation popover. In it, you will see a SQL builder UI and a dropdown menu of connections you can use to load the data into the chosen destination. You must choose a destination connection in order to build the data product.
Step 6: Paste or construct your SQL statement in the builder.
Step 7: Select "preview results". If your query is successful, you will see a preview of the data product in the table below the SQL builder.
Step 8: Once you are happy with your query, select Done
Step 9: Select "Create dataset" to run the query and load the data into the chosen destination.
Step 10: Verify the dataset and add metadata as needed.