Train-Test Split: Purpose, Methods, and Common Mistakes

How-to basics · Updated November 22, 2026

A train-test split function divides a dataset into separate portions: one for fitting a machine-learning model and another for evaluating it on data the model did not train on. In Python, scikit-learn’s train_test_split is a widely used function for making this division, with options for test size, randomization, stratification, and reproducibility.

What a train-test split is for

A model can often perform well on examples it has already seen, even if it has learned details that do not generalize. Measuring performance on the training data alone therefore gives an overly optimistic picture of how the model may behave on new cases. A held-out test set offers a more independent check: the model is fitted using the training portion, then evaluated on the test portion.

For example, imagine 1,000 labeled product reviews and a model intended to classify each review as positive or negative. A split might allocate 800 reviews for training and 200 for testing. The model learns from the 800, and its predictions for the 200 held-out reviews can be compared with their known labels. The test results estimate performance on similar, unseen reviews, not a guarantee of future accuracy.

The split is about evaluating a modeling process, not merely dividing rows for convenience. The test set should remain separate from decisions that shape the model, including choosing features, tuning parameters, and selecting among candidate models. Otherwise, information from the test set can influence the final system and make its reported score less trustworthy.

How the function works

In scikit-learn, train_test_split accepts one or more aligned arrays or similar indexable data structures. A common use is to pass a feature table, usually called X, and a target array, often called y. The function applies the same division to each input, so a feature row stays paired with its correct label.

The returned items alternate between training and test portions for each input. With X and y, the usual assignment is X_train, X_test, y_train, y_test. If neither test_size nor train_size is provided, scikit-learn uses a test portion of 25 percent. Setting one or both sizes makes the intended allocation explicit.

  • test_size can be a fraction, such as 0.2, or an integer number of rows.
  • train_size can likewise be a fraction or integer. If only one size is specified, the other portion uses the remaining data.
  • shuffle controls whether samples are shuffled before splitting; it is True by default.
  • random_state controls the randomization so a run can be repeated with the same data and settings.
  • stratify can preserve approximate class proportions in classification splits.

A basic scikit-learn example

For a classification dataset with features in X and class labels in y, this example reserves 20 percent for testing and requests a stratified split:

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

Here, random_state=42 is a reproducibility setting, not a special statistically correct number. Any fixed integer can serve the same basic purpose. Stratification is useful when classes are imbalanced; it aims to keep their proportions similar in the training and test portions. For instance, if 10 percent of the full dataset belongs to a minority class, a stratified test set will generally contain about that proportion, subject to the available row counts.

After splitting, fit the model with the training data and evaluate it using only the test data. For a classifier, the evaluation may include accuracy, precision, recall, F1 score, or a confusion matrix; the right measure depends on the task and the cost of different errors. A split function does not train a model or determine whether a metric is appropriate. It only creates the portions used in those later steps.

How to choose the split size

There is no universally best ratio. Common choices include 80/20 and 70/30, but the useful allocation depends on how much data is available, how variable the results are, and how much data the model needs to learn. With a very large dataset, a relatively small test fraction may still contain many representative cases. With a small dataset, holding out too much can leave too little data for fitting and make the test score unstable.

Think about the number of examples that will end up in the test set, not just the percentage. A test set with only a handful of rare positive cases cannot provide a precise estimate of performance on that class. Stratification can help distribute rare labels, but it cannot create examples that are absent, and it may fail or be unsuitable when a class has too few observations for the requested split.

When model selection is part of the work, a single train-test split is often not enough. A common approach is to make training, validation, and test portions: use training data to fit, validation data to compare models and tune settings, and test data for a final evaluation. Another option is cross-validation within the training data. Keep the test set out of repeated experimentation; repeatedly checking it and adjusting the model to improve its score gradually turns it into another validation set.

Randomness, stratification, and grouped data

Shuffling is appropriate when rows are independent and their order has no meaningful connection to the outcome. It can prevent an accidental split based on row order—for example, where the first half of a file comes from one class and the second half from another. A fixed random state makes the shuffled division repeatable, which helps compare model changes fairly. If the input data or its ordering changes, however, the resulting split may also change.

Stratification is mainly a classification technique. It aims to preserve the distribution of each label across portions, rather than selecting rows without regard to their class. It does not guarantee that every other characteristic is balanced, nor does it correct biased or unrepresentative source data. For regression, a continuous target is not ordinarily stratified directly; if target ranges matter, a practitioner may create justified bins for splitting, while taking care not to turn that convenience into a misleading evaluation design.

Some rows are related and must stay together. For example, multiple medical records may belong to one patient, several transactions to one customer, or many images to one subject. Randomly putting one person’s records in training and other records from that same person in testing can let the model benefit from subject-specific patterns. In these settings, use a group-aware splitting method, such as scikit-learn’s group splitters, so a group is kept together. The split should reflect how the model will encounter genuinely new cases.

Time-ordered data needs a different approach

For time series, forecasting, and other settings where the past is used to predict the future, a randomly shuffled split can produce an unrealistic test. It may train on later observations while testing on earlier ones, allowing information from the future to influence an evaluation meant to simulate forecasting. Instead, split by time: train on earlier periods and evaluate on a later period, sometimes using several rolling or expanding evaluation windows.

The exact design depends on the prediction task. If predictions are made one day ahead, the test period should represent that use case; if the model is refreshed periodically, the evaluation should reflect the refresh schedule. Pay attention to features too: a feature computed using future information can cause leakage even when rows are separated chronologically. A chronological split prevents some forms of leakage, but it does not automatically make every feature valid.

Common mistakes and how to avoid them

  • Preprocessing before splitting: A scaler, imputer, feature selector, or other learned transformation can capture information from the full dataset. Split first, fit such transformations on training data only, then apply them to validation and test data. Scikit-learn pipelines help enforce this sequence.
  • Splitting features and labels separately: Independent calls can put a feature row beside the wrong target. Pass aligned arrays together to the same call, or preserve a shared index when using another workflow.
  • Using an inappropriate random split: Repeated people, households, devices, or time periods may make rows dependent. Choose group-aware or chronological splitting when the data-generating structure calls for it.
  • Reusing the test set for tuning: Every decision based on test results leaks information into the model-selection process. Tune with a validation set or cross-validation, and reserve the test set for the final check.
  • Reporting a score without context: A score depends on the split, sample size, label balance, and metric. Report the evaluation setup and consider uncertainty or repeated validation rather than treating one number as a universal truth.

A related misconception is that a random split automatically makes an evaluation fair. It only randomizes the allocation under its assumptions. If the sample itself is biased, if observations are dependent, or if the future differs from the past, a random split can still yield a misleading result.

Train-test split compared with cross-validation

A holdout split is simple and relatively inexpensive: fit and evaluate using one division. Its result can depend noticeably on which observations land in the test portion, especially with limited data. Cross-validation addresses some of that variability by creating multiple folds; in k-fold cross-validation, the model is trained and evaluated across several different train/validation divisions, with each fold serving as validation once.

Cross-validation usually costs more computation because the model is fitted repeatedly. It is useful for comparing models or tuning settings on training data, but it does not eliminate the need for a final untouched test set when an independent final estimate is required. For grouped and time-based data, ordinary random k-fold validation may still be inappropriate; use a scheme that respects the relevant groups or chronology.

Frequently Asked Questions

What does the train-test split function do?
It divides aligned data into a training portion for fitting a model and a test portion for evaluating it on held-out examples. In scikit-learn, train_test_split can split features and labels together and offers controls for size, shuffling, stratification, and repeatability.

What is the usual train-test split ratio?
There is no mandatory ratio. 80/20 and 70/30 are common starting points, but dataset size, class frequency, model needs, and the evaluation goal should guide the choice. Ensure the test set contains enough relevant examples to make its metrics useful.

Why use random_state in train_test_split?
A fixed random_state makes the random split repeatable with the same input and settings. This is helpful for reproducible comparisons, but the particular integer does not make the split inherently better or more representative.

Should I use stratify=y?
For many classification problems, yes, especially when classes are imbalanced, because stratification aims to preserve class proportions. It is not a solution for dependent observations or time-based prediction, and it requires enough examples in each class to form the requested portions.

Can I use train_test_split for time-series data?
It can technically divide time-series rows, but its default shuffling is generally not suitable for forecasting evaluation. Use a chronological holdout or time-series validation strategy so training data precedes the periods used for evaluation.

Related

Try it: the Date Difference Calculator: days, months, years and weekdays between two dates, with the DATEDIF formulas.

More in How-to basics

Sources: Function behaviour and menu paths: Microsoft Support documentation for Excel. Method & sources

Spotted an error in this guide? Let us know.