As you work with your dataset, you might need some additional details on the columns you want to transform. You can find additional details about any column, like the statistical data, basic counts, percentages of values, etc. by accessing the Column Details from any column menu.
Step 1 - Access Column Details
To access, click on the Column Dropdown and find the Column Details option in the menu.
This opens the Column Details view which contains visual profiling information about the selected column, useful statistics, value summaries etc.
Note-
When you select a column, the Details panel on the right will show a preview of the Column data quality, column values analysis and patterns identified. This is a subset of the full column details that you will find in the Column Details view.
The Column Details shown are specific to the type of column selected, so the profiling data and histograms shown will vary.
Step 2 - See Column Details Overview
The Column Details open with the Overview tab in view.
This Overview section helps you review basic counts and percentages of values in the selected column.
Values -
This view shows you a summary of values, sectioned into Valid, Mismatched and Missing. It also shows the top values that appear in the column along with their frequencies.
Another chart shows the mismatched values and outliers along with their occurrences.
Statistics -
For a numeric column, you can see statistics about your column like minimum, maximum, average, median and quartile values.
For a String type column, this will show string length statistics such as maximum, minimum string lengths, etc.
Histograms -
You are shown Values Histogram and Frequent Values for integer type columns and String Length and Frequent Length histograms for String columns
For a Date type column, the profiling information will vary slightly. Although the Values and Statistics remain similar to integer values, the Histograms will show datetime-specific information like Month and Day Of Month along with the Value Histogram and Frequent Values histogram.
This visual profiling information puts you way ahead in your understanding and analysis of your column data by giving you a complete picture of the its values without scrolling through thousands of rows!
Step 3 - See Column Patterns and use them for your transformations
You can switch to Patterns view by clicking on the Patterns tab on top of the Column Details view.
Here you can review patterns identified by the platform in the column data and create steps based in these patterns.
Pattern profiling finds and groups the clusters of similar data values based on their format and structure such as differently formatted phone numbers, addresses, log entries and name fields.
All non-blank values are captured in the All Patterns category
Pattern suggestions are created based on the first few thousand rows of data in your sample.
When you select a pattern group, you may be presented with suggestions for standardising the values in the column to a single format.
Example of Patterns for Integer column-
Example of Patterns for Date type column-
How to Utilise Patterns for data transformation-
a) Select a pattern to trigger a set of suggestion cards to apply to the represented data. Click on empty area in the pattern to make the selection -
b) In the Suggestions panel on the right side, look for Convert and see the suggestions based on your selected pattern
c) It shows the affected column values and their updates based on pattern matching.
You can edit this transform in Transform Builder by clicking Edit
d) You can Add this step directly to your recipe.
More Info
To learn more about Column Details view, read this detailed documentation guide.
Depending on the data type of the column, additional statistics provide information on data quality and variation. For more information, see Column Statistics Reference.
For more information on pattern standardization, see Standardize Using Patterns.