> For the complete documentation index, see [llms.txt](https://docs.upriverdata.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.upriverdata.com/monitoring/field-behavior-monitoring.md).

# Field Behavior Monitoring

## Overview

Upriver provides a wide variety of monitors that are automatically set for every field in the data source. Users can also define custom monitors using Upriver's `Customer Metrics` (see [Custom Metrics](/monitoring/custom-metrics.md)).

For all of the monitored metrics, Upriver automatically sets the threshold expectations by analyzing the data and allows the users to set up thresholds on top of these values (see `User Exptations` in [Data Source Fields](/product-usage/data-assets/data-source/data-source-fields.md)).

## Monitored Metrics

### Field Metadata

| Metric                         | Description                                                                                                                                                                                                                                  |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Completeness (%)               | Percentage of rows where the field value is null or the field is not available (for semi-structured data).                                                                                                                                   |
| Default Value Completeness (%) | <p>Percentage of rows where the field value is null or the field is not available (for semi-structured data) when considering the default field value as empty.<br>e.g., the completeness when considering empty strings as null values.</p> |
| Null Count                     | Count of rows where the field value is null or the field is not available (for semi-structured data).                                                                                                                                        |
| Non-null count                 | Count of rows where the field value is not null.                                                                                                                                                                                             |
| Field Uniqueness (%)           | Percentage of unique values for a specific fields. Calculated as `cardinality / row_count`.                                                                                                                                                  |
| Row Uniqueness                 | Percentage of unique rows across all fields.                                                                                                                                                                                                 |
| Cardinality                    | Count of the different number of unique values seen for a given field.                                                                                                                                                                       |

### Field Data Monitors

<table><thead><tr><th>Metric</th><th width="324">Description</th><th>Relevant Field Type</th></tr></thead><tbody><tr><td>Mean</td><td>Average value across all rows.</td><td>Numeric</td></tr><tr><td>Standard Deviation</td><td>Standard deviation of values across all rows.</td><td>Numeric</td></tr><tr><td>Max Value</td><td>Maximum value across all rows.</td><td>Numeric</td></tr><tr><td>Min Value</td><td>Minimum value across all rows.</td><td>Numeric</td></tr><tr><td>Percentiles</td><td>Expected value for the following percentiles:<br>0 - 10, 20, 30, 40, 50, 60, 70, 80, 90 - 100.</td><td>Numeric</td></tr><tr><td>Sum</td><td>Sum of values across all rows.</td><td>Numeric</td></tr><tr><td>String Length (mean)</td><td>Average length across all rows.</td><td>String</td></tr><tr><td>String Length (standard deviation)</td><td>Standard deviation of lengths across all rows.</td><td>String</td></tr><tr><td>String Length (max)</td><td>Maximum string length across all rows.</td><td>String</td></tr><tr><td>String Length (min)</td><td>Minimum string length across all rows.</td><td>String</td></tr><tr><td>String Length Percentiles</td><td>Expected string lengths for the following percentiles:<br>0 - 10, 20, 30, 40, 50, 60, 70, 80, 90 - 100.</td><td>String</td></tr><tr><td>Array Size (mean)</td><td>Average array size across all rows.</td><td>Array</td></tr><tr><td>Array Size (standard deviation)</td><td>Standard deviation of array sizes across all rows.</td><td>Array</td></tr><tr><td>Array Size (max)</td><td>Maximum array size across all rows.</td><td>Array</td></tr><tr><td>Array Size (min)</td><td>Minimum array size across all rows.</td><td>Array</td></tr><tr><td>Array Size Percentiles</td><td>Expected array sizes for the following percentiles:<br>0 - 10, 20, 30, 40, 50, 60, 70, 80, 90 - 100.</td><td>Array</td></tr><tr><td>Value distribution (Histogram)</td><td><strong>This is available only for categorical fields</strong><br>The distribution between the different values seen for a given field.</td><td>Any</td></tr><tr><td>Semantic Format Distribution</td><td>The semantic formats seen for a given field (e.g. UUID, SSN, email, IP, etc.) and their percentage of rows in which the field adheres to each format.</td><td>Numeric, String</td></tr></tbody></table>

### Setting expectations on fields

To configure custom expectations per field, access "user expectations" from inside the main data source page.

<figure><img src="https://875415170-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FLE0zsJmUpeLjYw085cnO%2Fuploads%2FSpn7K4KKBYNsSPvMiB0O%2Fuser%20expec.jpg?alt=media&amp;token=9ed84680-d441-4591-9f3b-3c3ba1675e94" alt=""><figcaption></figcaption></figure>

Under user expectations, click on "+add expectations" to set a new one.

Choose your field, a metric to monitor, and the custom values. Save.

<figure><img src="https://875415170-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FLE0zsJmUpeLjYw085cnO%2Fuploads%2Fax79Zq8BUXIpiLFIorkx%2Fcompleteness-gif.gif?alt=media&amp;token=7321ac67-2661-478d-afbe-f256d7db4378" alt=""><figcaption></figcaption></figure>
