Skip to content

Machine Learning

11 built-in function(s) in this category.

  • Classify — Classify[data] trains a classifier and returns a ClassifierFunction. Data is a list of rules {features -> class, ...}; a class may be any expression -- a string, a symbol, a number -- and the distinct classes are numbered by first appearance. Method -> "NearestNeighbors" is the only method implemented and is the default, with NeighborsNumber defaulting to 1: a classifier votes rather than averages, so at k = 1 it reproduces its training labels exactly. Apply the result to a feature vector for a class, or with "Probabilities" for the vote shares. It also answers "Classes", "Method", "FeatureCount" and "NeighborCount". Method -> "NaiveBayes" fits a Gaussian per class with a diagonal covariance; Method -> "LogisticRegression" fits a logistic model by iteratively reweighted least squares with a small ridge on the non-intercept coefficients -- the ridge is load-bearing, because on linearly separable data the unpenalised likelihood is unbounded and the coefficients would diverge. Two classes give a single fit; more than two are fitted one-vs-rest, one binary model per class, and the class is the arg-max of the fitted probabilities. Those probabilities are normalised to sum to 1, which is a convention rather than a likelihood -- being monotone it cannot change the arg-max, so the class is the better-founded of the two answers. A single class declines: it is not a classification problem. (Stable)
  • DimensionReduce — DimensionReduce[data, k] reduces each row of data to k dimensions. Method -> "PrincipalComponentsAnalysis" (default) centres the columns and projects onto the leading eigenvectors of the covariance; "LatentSemanticAnalysis" skips the centring, giving a truncated SVD, which is what a sparse non-negative term-document matrix wants; "MultidimensionalScaling" double-centres the squared distance matrix (classical Torgerson scaling) and is capped at 2000 rows, its matrix being n x n. Asking for more dimensions than the data supports returns unevaluated rather than padding with zeros. (Stable)
  • DimensionReduction — DimensionReduction[data, k] returns a DimensionReducerFunction projecting into k dimensions, applicable to data it was NOT trained on -- the difference from DimensionReduce[data, k], which returns the reduced training data. New rows are centred on the TRAINING column means, which is what makes projections comparable across batches. Accepts one point or a matrix of points, and answers "Method", "FeatureCount" and "ReducedDimension". (Stable)
  • LearnDistribution — LearnDistribution[data] fits a distribution to data and returns a LearnedDistribution, usable with PDF. Method -> "Multinormal" is the default; Method -> "GaussianMixture" fits a mixture, choosing the component count by BIC. Multinormal fits a mean vector and a sample covariance (n-1 divisor, matching Variance). Rows are observations and columns are variables; a flat list is n observations of one variable. A singular covariance -- collinear columns, or fewer observations than dimensions -- returns unevaluated, because no density exists rather than because of an error. Method -> "ContingencyTable" is for NOMINAL data instead of numeric: it stores a probability per distinct outcome, which in one dimension is a categorical distribution. Outcomes may be any expressions -- strings, symbols, or equal-length lists of them -- compared structurally, and are kept in first-appearance order. Probabilities are empirical frequencies with no smoothing, so PDF of an outcome never observed is exactly 0; smoothing would require knowing how many outcomes were possible but unseen, which for arbitrary expressions is unknowable. Ragged outcomes decline. (Stable)
  • LinearModelFit — LinearModelFit[data] fits a linear model with an intercept and returns a PredictorFunction carrying its coefficients. Wolfram's version returns a FittedModel with regression diagnostics (RSquared, standard errors, ANOVA); those are not implemented and are not approximated. The coefficients are the same ones Predict finds. (Partial)
  • PDF — PDF[dist, x] gives the probability density of dist at x, and threads over a list of x. Supports NormalDistribution and UniformDistribution. (Stable)
  • Predict — Predict[data] fits a predictor to data and returns a PredictorFunction, which can be stored and applied to new inputs. Data is either a list of rules {features -> value, ...} or a matrix whose last column is the response. Method -> "LinearRegression" is the only method implemented and is the default; any other declines rather than silently linear-regressing. The returned object also answers "Method", "Coefficients" and "FeatureCount". A collinear feature set has no unique fit and returns unevaluated. (Stable)
  • PrincipalComponents — PrincipalComponents[matrix] gives the rows of matrix in principal-component coordinates, components ordered by decreasing variance. Rows are observations and columns are variables. Method -> "Correlation" standardises each variable to unit variance first, which is what you want when the columns have different units; the default "Covariance" does not. (Stable)
  • RandomVariate — RandomVariate[dist] draws one value from dist; RandomVariate[dist, n] draws a list of n. Supports NormalDistribution[mu, sigma] and UniformDistribution[{lo, hi}], each also usable with no arguments for the standard case. Draws come from the same stream as RandomReal, so SeedRandom makes them reproducible. A non-positive standard deviation, or an inverted range, returns unevaluated rather than producing NaNs. (Stable)
  • SmoothKernelDistribution — SmoothKernelDistribution[data] gives a kernel density estimate as a LearnedDistribution, usable with PDF. The kernel is a product Gaussian with a per-dimension bandwidth from the multivariate normal-reference rule, which in one dimension is Silverman's 1.06 sigma n^(-1/5). SmoothKernelDistribution[data, h] sets the bandwidth explicitly, as one number or one per dimension. Being a normal-reference rule the default oversmooths strongly multimodal data -- a known property of the rule, and the reason the explicit form exists. A constant column has no scale and returns unevaluated. (Stable)
  • Standardize — Standardize[data] shifts each column of data to zero mean and rescales it to unit sample standard deviation (divisor n-1, matching StandardDeviation). A flat list is treated as n observations of one variable. A constant column becomes exactly 0 rather than Indeterminate. (Stable)