DatasetReaderreads data from some location and constructs a
Dataset. All parameters necessary to read the data apart from the filepath should be passed to the constructor of the
from_params(params: allennlp.common.params.Params) → allennlp.data.dataset_readers.dataset_reader.DatasetReader¶
Static method that constructs the dataset reader described by
read(file_path: str) → allennlp.data.dataset.Dataset¶
Actually reads some data from the file_path and returns a
text_to_instance(*inputs) → allennlp.data.instance.Instance¶
Does whatever tokenization or processing is necessary to go from textual input to an
Instance. The primary intended use for this is with a
Predictor, which gets text input as a JSON object and needs to process it to be input to a model.
The intent here is to share code between
read()and what happens at model serving time, or any other time you want to make a prediction from new data. We need to process the data in the same way it was done at training time. Allowing the
DatasetReaderto process new text lets us accomplish this, as we can just call
DatasetReader.text_to_instancewhen serving predictions.
The input type here is rather vaguely specified, unfortunately. The
Predictorwill have to make some assumptions about the kind of
DatasetReaderthat it’s using, in order to pass it the right information.