allennlp.modules.text_field_embedders¶
A TextFieldEmbedder
is a Module
that takes as input the dict
of NumPy arrays
produced by a TextField
and
returns as output an embedded representation of the tokens in that field.
-
class
allennlp.modules.text_field_embedders.text_field_embedder.
TextFieldEmbedder
[source]¶ Bases:
torch.nn.modules.module.Module
,allennlp.common.registrable.Registrable
A
TextFieldEmbedder
is aModule
that takes as input theDataArray
produced by aTextField
and returns as output an embedded representation of the tokens in that field.The
DataArrays
produced byTextFields
are dictionaries with named representations, like “words” and “characters”. When you create aTextField
, you pass in a dictionary ofTokenIndexer
objects, telling the field how exactly the tokens in the field should be represented. This class changes the type signature ofModule.forward
, restrictingTextFieldEmbedders
to take inputs corresponding to a singleTextField
, which is a dictionary of tensors with the same names as were passed to theTextField
.We also add a method to the basic
Module
API:get_output_dim()
. You might need this if you want to construct aLinear
layer using the output of this embedder, for instance.-
default_implementation
= 'basic'¶
-
forward
(text_field_input: typing.Dict[str, torch.FloatTensor], num_wrapping_dims: int = 0) → torch.FloatTensor[source]¶ Parameters: text_field_input :
Dict[str, torch.Tensor]
A dictionary that was the output of a call to
TextField.as_tensor
. Each tensor in here is assumed to have a shape roughly similar to(batch_size, sequence_length)
(perhaps with an extra trailing dimension for the characters in each token).num_wrapping_dims :
int
, optional (default=0)If you have a
ListField[TextField]
that created thetext_field_input
, you’ll end up with tensors of shape(batch_size, wrapping_dim1, wrapping_dim2, ..., sequence_length)
. This parameter tells us how many wrapping dimensions there are, so that we can correctlyTimeDistribute
the embedding of each named representation.
-
-
class
allennlp.modules.text_field_embedders.basic_text_field_embedder.
BasicTextFieldEmbedder
(token_embedders: typing.Dict[str, allennlp.modules.token_embedders.token_embedder.TokenEmbedder]) → None[source]¶ Bases:
allennlp.modules.text_field_embedders.text_field_embedder.TextFieldEmbedder
This is a
TextFieldEmbedder
that wraps a collection ofTokenEmbedder
objects. EachTokenEmbedder
embeds or encodes the representation output from oneTokenIndexer
. As the data produced by aTextField
is a dictionary mapping names to these representations, we takeTokenEmbedders
with corresponding names. EachTokenEmbedders
embeds its input, and the result is concatenated in an arbitrary order.-
forward
(text_field_input: typing.Dict[str, torch.FloatTensor], num_wrapping_dims: int = 0) → torch.FloatTensor[source]¶
-