LINADForestHyperparameters

Properties

n_trees

integer | object≥ 1

Number of trees in the forest.

mtry_split

integer | object | null≥ 1

Features sampled at each split search. NULL scans every feature the tree holds. Cannot exceed the number of features in the training data.

mtry_tree

integer | object | null≥ 1

Features available to a whole tree, splits and node models alike. NULL gives every tree every feature. Cannot exceed the number of features in the training data.

max_leaves

integer | object≥ 1

Largest number of terminal nodes to grow. Plays the role the number of trees plays in gradient boosting.

patience

integer | object | null≥ 1

Expansions without validation improvement before growth stops, as gradient boosting stops on rounds. Needs a validation set; without one the tree grows to max_leaves. Stopping only bounds growth -- the size is still the one that minimizes validation loss over the curve reached. Defaults to growing to max_leaves.

force_max_leaves

boolean

Keep every leaf grown instead of selecting a tree size on the validation set.

smooth_validation_curve

boolean | null

Smooth the validation curve before reading its minimum. Steadier when the validation set is small. Applies only when force_max_leaves is FALSE.

min_cases_split

integer | object≥ 2

Fewest cases a node may hold and still be considered for splitting.

min_cases_leaf

integer | object≥ 1

Fewest cases a split must leave on each side.

min_cases_node_model

integer | object | null≥ 1

Fewest cases needed to fit a linear model at a node. Below it the node inherits its parent unchanged. Applies only when node_model is forward, ridge or elasticnet.

node_model

string

Model fitted at each node. constant is the intercept-only model every decision tree node carries; the others add a linear model on top of it, regularized by lambda.

one of"forward""ridge""elasticnet""constant"

nvmax

integer | object | null≥ 1

Most terms forward selection may add beside the intercept. A ceiling: the search stops earlier when forward_stop says a term does not pay for itself. Capped at the width of the design. Defaults to 3 where it applies. Applies only when node_model is forward.

forward_stop

string | object | null

Cost a term must earn to be added: the Bayesian or Akaike criterion, or none to always add nvmax terms. Shrinkage alone cannot stop the search, since every added term reduces the residual sum of squares. Defaults to bic where it applies. Applies only when node_model is forward.

one of"bic""aic""none"

lambda

number | object | null≥ 0

L2 penalty on the node models, on a standardized design so one value means the same at every node. Under forward selection it penalizes the fit and the search alike, so a feature that only looks good unpenalized does not win. Defaults to 0.05 where it applies. Applies only when node_model is forward, ridge or elasticnet.

alpha

number | object | null≥ 0≤ 1

Elastic-net mixing: 0 is ridge, 1 is lasso. Applies only when node_model is elasticnet.

learning_rate

number | object≤ 1> 0

Shrinkage applied to every functional update.

root_model

string | null

Model fitted at the root. NULL uses node_model.

one of"forward""ridge""elasticnet""constant"null

root_nvmax

integer | object | null≥ 1

nvmax for the root model. NULL uses nvmax. Ignored unless the root model is forward selection.

root_lambda

number | object | null≥ 0

lambda for the root model. NULL uses lambda. Ignored unless the root model is penalized.

root_alpha

number | object | null≥ 0≤ 1

alpha for the root model. NULL uses alpha. Ignored unless the root model is an elastic net.

root_learning_rate

number | object≥ 0≤ 1

Shrinkage applied to the root model's slopes. 0 fits no root model, so the first step is a split; 1 fits it in full.

node_test

string | object | null

Cost a node's slopes must earn over the constant alone, so that a node carries coefficients only where the data supports them and a plain constant otherwise. The constant is nested in the linear model, so on the node's own cases the slopes always fit better and no comparison is possible without a cost; aic charges 2 per nonzero slope and bic log(n). Forward selection has the same criterion per term in forward_stop, which subsumes this one, so this is the shrinking models' equivalent. Defaults to none where it applies. Applies only when node_model is ridge or elasticnet.

one of"none""aic""bic"

split_criterion

string | object | null

What the stump search scores a candidate side by: the level its fit explains, or its level and its slope in the split variable. linear finds a change of slope in the split variable, such as a parabola's vertex, which a level criterion cannot see; it can also chase the slope of the wrong variable where the change belongs to another. Defaults to mean where it applies. Applies only when split_search is stump.

one of"mean""linear"

split_search

string

How a split is chosen. stump takes the best squared-error split of the gradient; exhaustive scores each candidate by the loss after fitting both child models.

one of"stump""exhaustive"

split_binning

integer | object | null≥ 2

Discretize each numeric feature into this many equal-frequency bins and consider only bin boundaries as splits. Applies to both split searches. NULL considers every distinct value.

split_bin_type

string | object

How split_binning places bin edges: frequency spaces them evenly through the cases, width evenly through the feature's range.

one of"frequency""width"

n_cuts

integer | object | null≥ 2

Number of cut points tried per feature by the exhaustive search. split_bin_type decides how they are spaced. Applies only when split_search is exhaustive.

split_features

array | null≥ 1 items

Features that may define a split. NULL imposes no constraint, so every feature may. Independent of the linear roles: a feature can split without entering any node model, or the reverse. Values must name training features.

linear_features

array | null≥ 1 items

Features that get a slope in the node models. NULL imposes no constraint, so every feature does. A tree with no slopes at all is node_model = constant. Values must name training features.

global_features

array | null≥ 1 items

Linear features whose slope is shared by every leaf: the root estimates it and no node below may change it. Must be a subset of linear_features where that is set. NULL imposes no constraint, so no coefficient is pinned and every linear feature may adapt. Values must name training features.

gamma

number | object≥ 0≤ 1

Weight retained by a case in the branch it does not belong to. 0 is a hard partition; larger values share influence and reduce variance.

line_search

string | object

Scope of the Newton step for each update: one per expansion, one per child, or none.

one of"expansion""child""none"

line_search_max

number> 0

Largest absolute step the line search may take.

constant_rule

string | object

Rule for a node's constant: the manuscript's closed forms, or the weighted mean of the residual. Identical for regression; they differ only for classification.

one of"closed_form""least_squares"

ifw

boolean | object

Inverse Frequency Weighting in classification.

Relationships