# ⇅ Vector Features

Vector features enable providing an ordered set of numerical values within a single feature.

This is useful for providing pre-trained representations or activations obtained from other models or for providing multivariate inputs and outputs. An interesting use of vector features is the possibility of providing a probability distribution as output for a multiclass classification problem instead of a single correct class like with a category feature. Vector output features can also be useful for distillation and noise-aware losses.

### Vector Feature Preprocessing¶

The data is expected as whitespace separated numerical values. Example: "1.0 0.0 1.04 10.49". All vectors are expected to be of the same size.

Preprocessing parameters:

`vector_size`

(default`null`

): size of the vector. If not provided, it will be inferred from the data.`missing_value_strategy`

(default`fill_with_const`

): what strategy to follow when there's a missing value. The value should be one of`fill_with_const`

(replaces the missing value with a specific value specified with the`fill_value`

parameter),`fill_with_mode`

(replaces the missing values with the most frequent value in the column),`fill_with_mean`

(replaces the missing values with the mean of the values in the column),`bfill`

(replaces the missing values with the next valid value),`ffill`

(replaces the missing values with the previous valid value).`fill_value`

(default`""`

): the value to replace the missing values with in case the`missing_value_strategy`

is`fill_with_const`

.

Configuration example:

```
name: vector_feature_name
type: vector
preprocessing:
vector_size: null
missing_value_strategy: fill_with_const
fill_value: ""
```

Preprocessing parameters can also be defined once and applied to all vector input features using the Type-Global Preprocessing section.

### Vector Feature Encoders¶

The vector feature supports two encoders: `dense`

and `passthrough`

.

The encoder parameters specified at the feature level are:

`tied`

(default`null`

): name of the input feature to tie the weights of the encoder with. It needs to be the name of a feature of the same type and with the same encoder parameters.

Example vector feature entry in the input features list:

```
name: vector_column_name
type: vector
tied: null
encoder:
type: dense
```

The available encoder parameters are:

`type`

(default`dense`

): the possible values are`passthrough`

and`dense`

.`passthrough`

outputs the raw vector values unaltered.`dense`

uses a stack of fully connected layers to create an embedding matrix.

Encoder type and encoder parameters can also be defined once and applied to all vector input features using the Type-Global Encoder section.

#### Passthrough Encoder¶

There are no additional parameters for the `passthrough`

encoder.

#### Dense Encoder¶

For vector features, a dense encoder (stack of fully connected layers) can be used to encode the vector. It takes the following parameters:

`layers`

(default`null`

): a list of dictionaries containing the parameters of all the fully connected layers. The length of the list determines the number of stacked fully connected layers and the content of each dictionary determines the parameters for a specific layer. The available parameters for each layer are:`activation`

,`dropout`

,`norm`

,`norm_params`

,`output_size`

,`use_bias`

,`bias_initializer`

and`weights_initializer`

. If any of those values is missing from the dictionary, the default one specified as a parameter of the encoder will be used instead. If both`fc_layers`

and`num_fc_layers`

are`null`

, a default list will be assigned to`fc_layers`

with the value`[{output_size: 512}, {output_size: 256}]`

(only applies if`reduce_output`

is not`null`

).`num_layers`

(default`0`

): This is the number of stacked fully connected layers.`output_size`

(default`256`

): if a`output_size`

is not already specified in`fc_layers`

this is the default`output_size`

that will be used for each layer. It indicates the size of the output of a fully connected layer.`use_bias`

(default`true`

): boolean, whether the layer uses a bias vector.`weights_initializer`

(default`'glorot_uniform'`

): initializer for the weights matrix. Options are:`constant`

,`identity`

,`zeros`

,`ones`

,`orthogonal`

,`normal`

,`uniform`

,`truncated_normal`

,`variance_scaling`

,`glorot_normal`

,`glorot_uniform`

,`xavier_normal`

,`xavier_uniform`

,`he_normal`

,`he_uniform`

,`lecun_normal`

,`lecun_uniform`

.`bias_initializer`

(default`'zeros'`

): initializer for the bias vector. Options are:`constant`

,`identity`

,`zeros`

,`ones`

,`orthogonal`

,`normal`

,`uniform`

,`truncated_normal`

,`variance_scaling`

,`glorot_normal`

,`glorot_uniform`

,`xavier_normal`

,`xavier_uniform`

,`he_normal`

,`he_uniform`

,`lecun_normal`

,`lecun_uniform`

.`norm`

(default`null`

): if a`norm`

is not already specified in`fc_layers`

this is the default`norm`

that will be used for each layer. It indicates the norm of the output and it can be`null`

,`batch`

or`layer`

.`norm_params`

(default`null`

): parameters used if`norm`

is either`batch`

or`layer`

. For information on parameters used with`batch`

see Torch's documentation on batch normalization or for`layer`

see Torch's documentation on layer normalization.`activation`

(default`relu`

): if an`activation`

is not already specified in`fc_layers`

this is the default`activation`

that will be used for each layer. It indicates the activation function applied to the output.`dropout`

(default`0`

): dropout rate

Example vector feature entry in the input features list using a dense encoder:

```
name: vector_column_name
type: vector
tied: null
encoder:
type: dense
layers: null
num_layers: 0
output_size: 256
use_bias: true
weights_initializer: glorot_uniform
bias_initializer: zeros
norm: null
norm_params: null
activation: relu
dropout: 0
```

### Vector Feature Decoders¶

```
name: vector_column_name
type: vector
reduce_input: sum
dependencies: []
reduce_dependencies: sum
loss:
type: sigmoid_cross_entropy
decoder:
fc_layers: null
num_fc_layers: 0
output_size: 256
use_bias: true
weights_initializer: glorot_uniform
bias_initializer: zeros
activation: relu
clip: null
```

Vector features can be used when multi-class classification needs to be performed with a noise-aware loss or when the task is multivariate regression.

There is only one decoder available for vector features: a (potentially empty) stack of fully connected layers, followed by a projection into a tensor of the vector size (optionally followed by a softmax in the case of multi-class classification).

```
+--------------+ +---------+ +-----------+
|Combiner | |Fully | |Projection | +-------+
|Output +--->Connected+--->into Vector+--->Softmax|
|Representation| |Layers | |Size | +-------+
+--------------+ +---------+ +-----------+
```

These are the available parameters of the vector output feature.

`reduce_input`

(default`sum`

): defines how to reduce an input that is not a vector, but a matrix or a higher order tensor, on the first dimension (second if you count the batch dimension). Available values are:`sum`

,`mean`

or`avg`

,`max`

,`concat`

(concatenates along the first dimension),`last`

(returns the last vector of the first dimension).`dependencies`

(default`[]`

): the output features this one is dependent on. For a detailed explanation refer to Output Features Dependencies.`reduce_dependencies`

(default`sum`

): defines how to reduce the output of a dependent feature that is not a vector, but a matrix or a higher order tensor, on the first dimension (second if you count the batch dimension). Available values are:`sum`

,`mean`

or`avg`

,`max`

,`concat`

(concatenates along the first dimension),`last`

(returns the last vector of the first dimension).`softmax`

(default`false`

): determines if to apply a softmax at the end of the decoder. It is useful for predicting a vector of values that sum up to 1 and can be interpreted as probabilities.`loss`

(default`{type: mean_squared_error}`

): is a dictionary containing a loss`type`

. The available loss`type`

are`mean_squared_error`

,`mean_absolute_error`

and`softmax_cross_entropy`

(use it only if`softmax`

is`true`

).

Loss type and loss related parameters can also be defined once and applied to all vector output features using the Type-Global Loss section.

These are the available parameters of a vector output feature decoder.

`fc_layers`

(default`null`

): a list of dictionaries containing the parameters of all the fully connected layers. The length of the list determines the number of stacked fully connected layers and the content of each dictionary determines the parameters for a specific layer. The available parameters for each layer are:`activation`

,`dropout`

,`norm`

,`norm_params`

,`output_size`

,`use_bias`

,`bias_initializer`

and`weights_initializer`

. If any of those values is missing from the dictionary, the default one specified as a parameter of the decoder will be used instead.`num_fc_layers`

(default 0): this is the number of stacked fully connected layers that the input to the feature passes through. Their output is projected in the feature's output space.`output_size`

(default`256`

): if a`output_size`

is not already specified in`fc_layers`

this is the default`output_size`

that will be used for each layer. It indicates the size of the output of a fully connected layer.`use_bias`

(default`true`

): boolean, whether the layer uses a bias vector.`weights_initializer`

(default`'glorot_uniform'`

): initializer for the weights matrix. Options are:`constant`

,`identity`

,`zeros`

,`ones`

,`orthogonal`

,`normal`

,`uniform`

,`truncated_normal`

,`variance_scaling`

,`glorot_normal`

,`glorot_uniform`

,`xavier_normal`

,`xavier_uniform`

,`he_normal`

,`he_uniform`

,`lecun_normal`

,`lecun_uniform`

.`bias_initializer`

(default`'zeros'`

): initializer for the bias vector. Options are:`constant`

,`identity`

,`zeros`

,`ones`

,`orthogonal`

,`normal`

,`uniform`

,`truncated_normal`

,`variance_scaling`

,`glorot_normal`

,`glorot_uniform`

,`xavier_normal`

,`xavier_uniform`

,`he_normal`

,`he_uniform`

,`lecun_normal`

,`lecun_uniform`

.`activation`

(default`relu`

): if an`activation`

is not already specified in`fc_layers`

this is the default`activation`

that will be used for each layer. It indicates the activation function applied to the output.`clip`

(default`null`

): If not`null`

it specifies a minimum and maximum value the predictions will be clipped to. The value can be either a list or a tuple of length 2, with the first value representing the minimum and the second the maximum. For instance`(-5,5)`

will make it so that all predictions will be clipped in the`[-5,5]`

interval.

Decoder type and decoder related parameters can also be defined once and applied to all vector output features using the Type-Global Decoder section.

### Vector Features Measures¶

The metrics that are calculated every epoch and are available for set features are `mean_squared_error`

, `mean_absolute_error`

, `r2`

, and the `loss`

itself.

You can set any of them as `validation_metric`

in the `training`

section of the configuration if you set the
`validation_field`

to be the name of a vector feature.