Utilities for working with the local dataset cache.

allennlp.common.file_utils.cached_path(url_or_filename: str, cache_dir: str = None) → str[source]

Given something that might be a URL (or might be a local path), determine which. If it’s a URL, download the file and cache it, and return the path to the cached file. If it’s already a local path, make sure the file exists and then return the path.

allennlp.common.file_utils.filename_to_url(filename: str) → typing.Tuple[str, str][source]

Recovers the the url from the encoded filename. Returns it and the ETag (which may be None)

allennlp.common.file_utils.get_from_cache(url: str, cache_dir: str = None) → str[source]

Given a URL, look for the corresponding dataset in the local cache. If it’s not there, download it. Then return the path to the cached file.

allennlp.common.file_utils.url_to_filename(url: str, etag: str = None) → str[source]

Converts a url into a filename in a reversible way. If etag is specified, add it on the end, separated by a period (which necessarily won’t appear in the base64-encoded filename). Get rid of the quotes in the etag, since Windows doesn’t like them.