allennlp.modules.token_embedders.pretrained_transformer_mismatched_embedder#

PretrainedTransformerMismatchedEmbedder#

PretrainedTransformerMismatchedEmbedder(
    self,
    model_name: str,
    max_length: int = None,
) -> None

Use this embedder to embed wordpieces given by PretrainedTransformerMismatchedIndexer and to pool the resulting vectors to get word-level representations.

Parameters

  • model_name : str The name of the transformers model to use. Should be the same as the corresponding PretrainedTransformerMismatchedIndexer.
  • max_length : int, optional (default = None) If positive, folds input token IDs into multiple segments of this length, pass them through the transformer model independently, and concatenate the final representations. Should be set to the same value as the max_length option on the PretrainedTransformerMismatchedIndexer.

forward#

PretrainedTransformerMismatchedEmbedder.forward(
    self,
    token_ids: torch.LongTensor,
    mask: torch.LongTensor,
    offsets: torch.LongTensor,
    wordpiece_mask: torch.LongTensor,
    type_ids: Optional[torch.LongTensor] = None,
    segment_concat_mask: Optional[torch.LongTensor] = None,
) -> torch.Tensor

Parameters

  • token_ids: torch.LongTensor
  • Shape: [batch_size, num_wordpieces] (for exception see PretrainedTransformerEmbedder).
  • mask: torch.LongTensor
  • Shape: [batch_size, num_orig_tokens].
  • offsets: torch.LongTensor
  • Shape: [batch_size, num_orig_tokens, 2]. Maps indices for the original tokens, i.e. those given as input to the indexer,
  • __to a span in token_ids. token_ids[i][offsets[i][j][0]__:offsets[i][j][1] + 1] corresponds to the original j-th token from the i-th batch.
  • wordpiece_mask: torch.LongTensor
  • Shape: [batch_size, num_wordpieces].
  • type_ids: Optional[torch.LongTensor]
  • Shape: [batch_size, num_wordpieces].
  • segment_concat_mask: Optional[torch.LongTensor] See PretrainedTransformerEmbedder.

Returns:

Shape: [batch_size, num_orig_tokens, embedding_size].

get_output_dim#

PretrainedTransformerMismatchedEmbedder.get_output_dim(self)

Returns the final output dimension that this TokenEmbedder uses to represent each token. This is not the shape of the returned tensor, but the last element of that shape.