Data for AI

A dataset contains examples collected for a task, with each example represented in a way that supports analysis, training, or evaluation. For supervised learning, labels need to match the target you intend to predict; proxy labels should be examined to see whether they really stand in for that target. Check for missing or invalid values, inconsistent units, duplicates, and mislabeled examples, and record corrections.

Separate training, validation, and test data before development decisions accumulate. Use training data to fit the model, validation data to guide choices, and the test set for a final estimate. Duplicates or information unavailable at prediction time can make evaluation misleading. The split method should reflect how the system will encounter examples, such as separating related records or respecting time order when appropriate.

• Machine Learning Glossary: ML Fundamentals

  Concise reference definitions for training, validation, and test sets.

• Data leakage

  A practical reference for recognizing and avoiding leakage during preprocessing and evaluation.

Dataset

A collection of examples used for analysis, training, or evaluation.

Training set

The examples used to adjust a model.

Validation set

Examples used to choose settings or compare development decisions.

Test set

Held-out examples used to estimate performance after development choices.

Data leakage

Information unavailable at deployment improperly enters training or evaluation.

Synthetic data

Artificially generated examples used instead of or alongside observed data.

Data provenance

The recorded origin, permissions, and transformation history of data.

Before modeling, inspect whether values make sense for the task and whether the collection process could have shaped the examples. Common problems include missing values, inconsistent units, invalid values, spelling variants, and mislabeling. Define checks for expected ranges and categories, then review errors and data slices that could be hidden by overall counts.

Labels also need scrutiny. A direct label matches the prediction target; a proxy label is only an approximation and may encode a different process. Write down labeling instructions and compare a sample of labels with careful human review. When correcting data or labels, record what changed and why so another analyst can trace how the working dataset differs from what was collected.

• Datasets: Labels

  Explains direct and proxy labels, human labeling, and label quality checks.

• Production ML systems: Monitoring pipelines

  Shows how schemas and feature checks can catch unexpected values and distributions.

For each dataset version, document where the data came from, how it was collected, what permissions or restrictions apply, what transformations were made, and what uses are appropriate. A structured record such as a datasheet helps creators and users discuss a dataset's composition, collection, recommended uses, and limitations. Provenance records support traceability; they do not establish that every value is correct.

Synthetic examples can supplement observed data or stand in for it, but the word synthetic alone does not establish privacy or usefulness. Compare generated data with the intended task, check how well it represents important groups and patterns, and assess disclosure risks when it was generated from identifiable data. A privacy claim requires evidence about the generation method and its guarantees, not just a synthetic label.

• Guidelines for Evaluating Differential Privacy Guarantees

  Discusses privacy and utility considerations for synthetic data, including risks when differential privacy is not used.

• SP 800-188, De-Identifying Government Datasets: Techniques and Governance

  A governance-oriented reference for considering data-sharing models and re-identification risk.