allennlp.modules.text_field_embedders

A TextFieldEmbedder is a Module that takes as input the dict of NumPy arrays produced by a TextField and returns as output an embedded representation of the tokens in that field.

class allennlp.modules.text_field_embedders.text_field_embedder.TextFieldEmbedder[source]

Bases: torch.nn.modules.module.Module, allennlp.common.registrable.Registrable

A TextFieldEmbedder is a Module that takes as input the DataArray produced by a TextField and returns as output an embedded representation of the tokens in that field.

The DataArrays produced by TextFields are dictionaries with named representations, like “words” and “characters”. When you create a TextField, you pass in a dictionary of TokenIndexer objects, telling the field how exactly the tokens in the field should be represented. This class changes the type signature of Module.forward, restricting TextFieldEmbedders to take inputs corresponding to a single TextField, which is a dictionary of tensors with the same names as were passed to the TextField.

We also add a method to the basic Module API: get_output_dim(). You might need this if you want to construct a Linear layer using the output of this embedder, for instance.

default_implementation = 'basic'
forward(text_field_input: typing.Dict[str, torch.FloatTensor]) → torch.FloatTensor[source]
classmethod from_params(vocab: allennlp.data.vocabulary.Vocabulary, params: allennlp.common.params.Params) → allennlp.modules.text_field_embedders.text_field_embedder.TextFieldEmbedder[source]
get_output_dim() → int[source]

Returns the dimension of the vector representing each token in the output of this TextFieldEmbedder. This is not the shape of the returned tensor, but the last element of that shape.

class allennlp.modules.text_field_embedders.basic_text_field_embedder.BasicTextFieldEmbedder(token_embedders: typing.Dict[str, allennlp.modules.token_embedders.token_embedder.TokenEmbedder]) → None[source]

Bases: allennlp.modules.text_field_embedders.text_field_embedder.TextFieldEmbedder

This is a TextFieldEmbedder that wraps a collection of TokenEmbedder objects. Each TokenEmbedder embeds or encodes the representation output from one TokenIndexer. As the data produced by a TextField is a dictionary mapping names to these representations, we take TokenEmbedders with corresponding names. Each TokenEmbedders embeds its input, and the result is concatenated in an arbitrary order.

forward(text_field_input: typing.Dict[str, torch.FloatTensor]) → torch.FloatTensor[source]
classmethod from_params(vocab: allennlp.data.vocabulary.Vocabulary, params: allennlp.common.params.Params) → allennlp.modules.text_field_embedders.basic_text_field_embedder.BasicTextFieldEmbedder[source]
get_output_dim() → int[source]

Returns the dimension of the vector representing each token in the output of this TextFieldEmbedder. This is not the shape of the returned tensor, but the last element of that shape.