# Vector Features

Vector features allow to provide an ordered set of numerical values all at once. This is useful for providing pre-trained representations or activations obtained from other models or for providing multivariate inputs and outputs. An interesting use of vector features is the possibility to provide a probability distribution as output for a multiclass classification problem instead of just the correct class like it is possible to do with category features. This is useful for distillation and noise-aware losses.

### Vector Feature Preprocessing¶

The data is expected as whitespace separated numerical values. Example: "1.0 0.0 1.04 10.49". All vectors are expected to be of the same size.

Preprocessing parameters:

`vector_size`

(default`null`

): size of the vector. If not provided, it will be inferred from the data.`missing_value_strategy`

(default`fill_with_const`

): what strategy to follow when there's a missing value. The value should be one of`fill_with_const`

(replaces the missing value with a specific value specified with the`fill_value`

parameter),`fill_with_mode`

(replaces the missing values with the most frequent value in the column),`fill_with_mean`

(replaces the missing values with the mean of the values in the column),`backfill`

(replaces the missing values with the next valid value).`fill_value`

(default ""): the value to replace the missing values with in case the`missing_value_strategy`

is`fill_value`

.

### Vector Feature Encoders¶

The vector feature supports two encoders: `dense`

and `passthrough`

. Only the `dense`

encoder has additional parameters, which is shown next.

#### Dense Encoder¶

For vector features, you can use a dense encoder (stack of fully connected layers). It takes the following parameters:

`layers`

(default`null`

): it is a list of dictionaries containing the parameters of all the fully connected layers. The length of the list determines the number of stacked fully connected layers and the content of each dictionary determines the parameters for a specific layer. The available parameters for each layer are:`fc_size`

,`norm`

,`activation`

and`regularize`

. If any of those values is missing from the dictionary, the default one specified as a parameter of the encoder will be used instead. If both`fc_layers`

and`num_fc_layers`

are`null`

, a default list will be assigned to`fc_layers`

with the value`[{fc_size: 512}, {fc_size: 256}]`

(only applies if`reduce_output`

is not`null`

).`num_layers`

(default`0`

): This is the number of stacked fully connected layers.`fc_size`

(default`256`

): if a`fc_size`

is not already specified in`fc_layers`

this is the default`fc_size`

that will be used for each layer. It indicates the size of the output of a fully connected layer.`use_bias`

(default`true`

): boolean, whether the layer uses a bias vector.`weights_initializer`

(default`'glorot_uniform'`

): initializer for the weights matrix. Options are:`constant`

,`identity`

,`zeros`

,`ones`

,`orthogonal`

,`normal`

,`uniform`

,`truncated_normal`

,`variance_scaling`

,`glorot_normal`

,`glorot_uniform`

,`xavier_normal`

,`xavier_uniform`

,`he_normal`

,`he_uniform`

,`lecun_normal`

,`lecun_uniform`

. Alternatively it is possible to specify a dictionary with a key`type`

that identifies the type of initializer and other keys for its parameters, e.g.`{type: normal, mean: 0, stddev: 0}`

. To know the parameters of each initializer, please refer to TensorFlow's documentation.`bias_initializer`

(default`'zeros'`

): initializer for the bias vector. Options are:`constant`

,`identity`

,`zeros`

,`ones`

,`orthogonal`

,`normal`

,`uniform`

,`truncated_normal`

,`variance_scaling`

,`glorot_normal`

,`glorot_uniform`

,`xavier_normal`

,`xavier_uniform`

,`he_normal`

,`he_uniform`

,`lecun_normal`

,`lecun_uniform`

. Alternatively it is possible to specify a dictionary with a key`type`

that identifies the type of initializer and other keys for its parameters, e.g.`{type: normal, mean: 0, stddev: 0}`

. To know the parameters of each initializer, please refer to TensorFlow's documentation.`weights_regularizer`

(default`null`

): regularizer function applied to the weights matrix. Valid values are`l1`

,`l2`

or`l1_l2`

.`bias_regularizer`

(default`null`

): regularizer function applied to the bias vector. Valid values are`l1`

,`l2`

or`l1_l2`

.`activity_regularizer`

(default`null`

): regurlizer function applied to the output of the layer. Valid values are`l1`

,`l2`

or`l1_l2`

.`norm`

(default`null`

): if a`norm`

is not already specified in`fc_layers`

this is the default`norm`

that will be used for each layer. It indicates the norm of the output and it can be`null`

,`batch`

or`layer`

.`norm_params`

(default`null`

): parameters used if`norm`

is either`batch`

or`layer`

. For information on parameters used with`batch`

see Tensorflow's documentation on batch normalization or for`layer`

see Tensorflow's documentation on layer normalization.`activation`

(default`relu`

): if an`activation`

is not already specified in`fc_layers`

this is the default`activation`

that will be used for each layer. It indicates the activation function applied to the output.`dropout`

(default`0`

): dropout rate

Example vector feature entry in the input features list using an dense encoder:

```
name: vector_column_name
type: vector
encoder: dense
layers: null
num_layers: 0
fc_size: 256
use_bias: true
weights_initializer: glorot_uniform
bias_initializer: zeros
weights_regularizer: null
bias_regularizer: null
activity_regularizer: null
norm: null
norm_params: null
activation: relu
dropout: 0
```

### Vector Feature Decoders¶

Vector features can be used when multi-class classification needs to be performed with a noise-aware loss or when the task is multivariate regression. There is only one decoder available for set features and it is a (potentially empty) stack of fully connected layers, followed by a projection into a vector of size (optionally followed by a softmax in the case of multi-class classification).

```
+--------------+ +---------+ +-----------+
|Combiner | |Fully | |Projection | +------------------+
|Output +--->Connected+--->into Output+--->Softmax (optional)|
|Representation| |Layers | |Space | +------------------+
+--------------+ +---------+ +-----------+
```

These are the available parameters of the set output feature

`reduce_input`

(default`sum`

): defines how to reduce an input that is not a vector, but a matrix or a higher order tensor, on the first dimension (second if you count the batch dimension). Available values are:`sum`

,`mean`

or`avg`

,`max`

,`concat`

(concatenates along the first dimension),`last`

(returns the last vector of the first dimension).`dependencies`

(default`[]`

): the output features this one is dependent on. For a detailed explanation refer to Output Features Dependencies.`reduce_dependencies`

(default`sum`

): defines how to reduce the output of a dependent feature that is not a vector, but a matrix or a higher order tensor, on the first dimension (second if you count the batch dimension). Available values are:`sum`

,`mean`

or`avg`

,`max`

,`concat`

(concatenates along the first dimension),`last`

(returns the last vector of the first dimension).`softmax`

(default`false`

): determines if to apply a softmax at the end of the decoder. It is useful for predicting a vector of values that sum up to 1 and can be interpreted as probabilities.`loss`

(default`{type: mean_squared_error}`

): is a dictionary containing a loss`type`

. The available loss`type`

are`mean_squared_error`

,`mean_absolute_error`

and`softmax_cross_entropy`

(use it only if`softmax`

is`true`

).

These are the available parameters of a set output feature decoder

`fc_layers`

(default`null`

): it is a list of dictionaries containing the parameters of all the fully connected layers. The length of the list determines the number of stacked fully connected layers and the content of each dictionary determines the parameters for a specific layer. The available parameters for each layer are:`fc_size`

,`norm`

,`activation`

,`dropout`

,`initializer`

and`regularize`

. If any of those values is missing from the dictionary, the default one specified as a parameter of the decoder will be used instead.`num_fc_layers`

(default 0): this is the number of stacked fully connected layers that the input to the feature passes through. Their output is projected in the feature's output space.`fc_size`

(default`256`

): if a`fc_size`

is not already specified in`fc_layers`

this is the default`fc_size`

that will be used for each layer. It indicates the size of the output of a fully connected layer.`use_bias`

(default`true`

): boolean, whether the layer uses a bias vector.`weights_initializer`

(default`'glorot_uniform'`

): initializer for the weights matrix. Options are:`constant`

,`identity`

,`zeros`

,`ones`

,`orthogonal`

,`normal`

,`uniform`

,`truncated_normal`

,`variance_scaling`

,`glorot_normal`

,`glorot_uniform`

,`xavier_normal`

,`xavier_uniform`

,`he_normal`

,`he_uniform`

,`lecun_normal`

,`lecun_uniform`

. Alternatively it is possible to specify a dictionary with a key`type`

that identifies the type of initializer and other keys for its parameters, e.g.`{type: normal, mean: 0, stddev: 0}`

. To know the parameters of each initializer, please refer to TensorFlow's documentation.`bias_initializer`

(default`'zeros'`

): initializer for the bias vector. Options are:`constant`

,`identity`

,`zeros`

,`ones`

,`orthogonal`

,`normal`

,`uniform`

,`truncated_normal`

,`variance_scaling`

,`glorot_normal`

,`glorot_uniform`

,`xavier_normal`

,`xavier_uniform`

,`he_normal`

,`he_uniform`

,`lecun_normal`

,`lecun_uniform`

. Alternatively it is possible to specify a dictionary with a key`type`

that identifies the type of initializer and other keys for its parameters, e.g.`{type: normal, mean: 0, stddev: 0}`

. To know the parameters of each initializer, please refer to TensorFlow's documentation.`weights_regularizer`

(default`null`

): regularizer function applied to the weights matrix. Valid values are`l1`

,`l2`

or`l1_l2`

.`bias_regularizer`

(default`null`

): regularizer function applied to the bias vector. Valid values are`l1`

,`l2`

or`l1_l2`

.`activity_regularizer`

(default`null`

): regurlizer function applied to the output of the layer. Valid values are`l1`

,`l2`

or`l1_l2`

.`activation`

(default`relu`

): if an`activation`

is not already specified in`fc_layers`

this is the default`activation`

that will be used for each layer. It indicates the activation function applied to the output.`clip`

(default`null`

): If not`null`

it specifies a minimum and maximum value the predictions will be clipped to. The value can be either a list or a tuple of length 2, with the first value representing the minimum and the second the maximum. For instance`(-5,5)`

will make it so that all predictions will be clipped in the`[-5,5]`

interval.

Example vector feature entry (with default parameters) in the output features list:

```
name: vector_column_name
type: vector
reduce_input: sum
dependencies: []
reduce_dependencies: sum
loss:
type: sigmoid_cross_entropy
fc_layers: null
num_fc_layers: 0
fc_size: 256
use_bias: true
weights_initializer: glorot_uniform
bias_initializer: zeros
weights_regularizer: null
bias_regularizer: null
activity_regularizer: null
activation: relu
clip: null
```

### Vector Features Measures¶

The measures that are calculated every epoch and are available for numerical features are `mean_squared_error`

, `mean_absolute_error`

, `r2`

and the `loss`

itself.
You can set either of them as `validation_measure`

in the `training`

section of the configuration if you set the `validation_field`

to be the name of a numerical feature.