Skip to content

rslearn.dataset.window_data_storage.storage

storage

Abstract classes for materialized window data storage.

LayerWriter

Bases: ABC

A layer-scoped writer used as a context manager.

Returned by :meth:WindowDataStorage.open_layer_writer. Callers iterate over item groups, calling :meth:write_raster / :meth:write_vector, and the writer's __exit__ flushes any buffered data.

Source code in rslearn/dataset/window_data_storage/storage.py 
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
class LayerWriter(ABC):
    """A layer-scoped writer used as a context manager.

    Returned by :meth:`WindowDataStorage.open_layer_writer`. Callers iterate
    over item groups, calling :meth:`write_raster` / :meth:`write_vector`,
    and the writer's ``__exit__`` flushes any buffered data.
    """

    @abstractmethod
    def write_raster(
        self,
        bands: list[str],
        raster_format: RasterFormat,
        projection: Projection,
        bounds: PixelBounds,
        raster: RasterArray,
        group_idx: int = 0,
    ) -> None:
        """Write a single item group's raster for one band set."""

    @abstractmethod
    def write_vector(
        self,
        vector_format: VectorFormat,
        features: list[Feature],
        group_idx: int = 0,
    ) -> None:
        """Write a single item group's vector features."""

    def __enter__(self) -> LayerWriter:
        """Return ``self`` so the writer can be used as a context manager."""
        return self

    @abstractmethod
    def __exit__(self, exc_type: Any, exc: Any, tb: Any) -> None:
        """Flush any buffered data on success.

        Implementations should not flush if ``exc_type`` is not ``None``.
        """

write_raster abstractmethod 

write_raster(bands: list[str], raster_format: RasterFormat, projection: Projection, bounds: PixelBounds, raster: RasterArray, group_idx: int = 0) -> None

Write a single item group's raster for one band set.

Source code in rslearn/dataset/window_data_storage/storage.py 
28
29
30
31
32
33
34
35
36
37
38
@abstractmethod
def write_raster(
    self,
    bands: list[str],
    raster_format: RasterFormat,
    projection: Projection,
    bounds: PixelBounds,
    raster: RasterArray,
    group_idx: int = 0,
) -> None:
    """Write a single item group's raster for one band set."""

write_vector abstractmethod 

write_vector(vector_format: VectorFormat, features: list[Feature], group_idx: int = 0) -> None

Write a single item group's vector features.

Source code in rslearn/dataset/window_data_storage/storage.py 
40
41
42
43
44
45
46
47
@abstractmethod
def write_vector(
    self,
    vector_format: VectorFormat,
    features: list[Feature],
    group_idx: int = 0,
) -> None:
    """Write a single item group's vector features."""

WindowDataStorage

Bases: ABC

Storage backend for per-window materialized raster and vector data.

A WindowDataStorage is bound to a specific window. It is created by a :class:WindowDataStorageFactory and holds a reference to its window.

Source code in rslearn/dataset/window_data_storage/storage.py 
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
class WindowDataStorage(ABC):
    """Storage backend for per-window materialized raster and vector data.

    A WindowDataStorage is bound to a specific window. It is created by a
    :class:`WindowDataStorageFactory` and holds a reference to its window.
    """

    def __init__(self, window: Window) -> None:
        """Initialize the storage bound to a specific window.

        Args:
            window: the window this storage is bound to.
        """
        self.window = window

    @abstractmethod
    def open_layer_writer(
        self,
        layer_name: str,
    ) -> LayerWriter:
        """Open a writer for one materialization pass over a layer.

        Args:
            layer_name: the layer name.
        """

    @abstractmethod
    def read_raster(
        self,
        layer_name: str,
        bands: list[str],
        raster_format: RasterFormat,
        projection: Projection | None = None,
        bounds: PixelBounds | None = None,
        group_idx: int = 0,
        resampling: Resampling = Resampling.bilinear,
    ) -> RasterArray:
        """Read a single item group's raster.

        Args:
            layer_name: the layer name.
            bands: the band set to read.
            raster_format: the raster format to decode with.
            projection: target projection (defaults to window projection).
            bounds: target bounds (defaults to window bounds).
            group_idx: the item group index (default 0).
            resampling: resampling method (defaults to bilinear).
        """

    def read_rasters(
        self,
        layer_name: str,
        bands: list[str],
        group_idxs: list[int],
        raster_format: RasterFormat,
        projection: Projection | None = None,
        bounds: PixelBounds | None = None,
        resampling: Resampling = Resampling.bilinear,
    ) -> list[RasterArray]:
        """Read rasters for the specified item groups.

        The default implementation loops over :meth:`read_raster`.
        """
        return [
            self.read_raster(
                layer_name,
                bands,
                raster_format,
                projection,
                bounds,
                group_idx=group_idx,
                resampling=resampling,
            )
            for group_idx in group_idxs
        ]

    @abstractmethod
    def read_vector(
        self,
        layer_name: str,
        vector_format: VectorFormat,
        projection: Projection | None = None,
        bounds: PixelBounds | None = None,
        group_idx: int = 0,
    ) -> list[Feature]:
        """Read a single item group's vector features.

        Args:
            layer_name: the layer name.
            vector_format: the vector format to decode with.
            projection: target projection (defaults to window projection).
            bounds: target bounds (defaults to window bounds).
            group_idx: the item group index (default 0).
        """

open_layer_writer abstractmethod 

open_layer_writer(layer_name: str) -> LayerWriter

Open a writer for one materialization pass over a layer.

Parameters:

Name Type Description Default
layer_name str

the layer name.

 required
Source code in rslearn/dataset/window_data_storage/storage.py 
76
77
78
79
80
81
82
83
84
85
@abstractmethod
def open_layer_writer(
    self,
    layer_name: str,
) -> LayerWriter:
    """Open a writer for one materialization pass over a layer.

    Args:
        layer_name: the layer name.
    """

read_raster abstractmethod 

read_raster(layer_name: str, bands: list[str], raster_format: RasterFormat, projection: Projection | None = None, bounds: PixelBounds | None = None, group_idx: int = 0, resampling: Resampling = bilinear) -> RasterArray

Read a single item group's raster.

Parameters:

Name Type Description Default
layer_name str

the layer name.

 required
bands list[str]

the band set to read.

 required
raster_format RasterFormat

the raster format to decode with.

 required
projection Projection | None

target projection (defaults to window projection).

 None
bounds PixelBounds | None

target bounds (defaults to window bounds).

 None
group_idx int

the item group index (default 0).

 0
resampling Resampling

resampling method (defaults to bilinear).

 bilinear
Source code in rslearn/dataset/window_data_storage/storage.py 
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
@abstractmethod
def read_raster(
    self,
    layer_name: str,
    bands: list[str],
    raster_format: RasterFormat,
    projection: Projection | None = None,
    bounds: PixelBounds | None = None,
    group_idx: int = 0,
    resampling: Resampling = Resampling.bilinear,
) -> RasterArray:
    """Read a single item group's raster.

    Args:
        layer_name: the layer name.
        bands: the band set to read.
        raster_format: the raster format to decode with.
        projection: target projection (defaults to window projection).
        bounds: target bounds (defaults to window bounds).
        group_idx: the item group index (default 0).
        resampling: resampling method (defaults to bilinear).
    """

read_rasters 

read_rasters(layer_name: str, bands: list[str], group_idxs: list[int], raster_format: RasterFormat, projection: Projection | None = None, bounds: PixelBounds | None = None, resampling: Resampling = bilinear) -> list[RasterArray]

Read rasters for the specified item groups.

The default implementation loops over :meth:read_raster.

Source code in rslearn/dataset/window_data_storage/storage.py 
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
def read_rasters(
    self,
    layer_name: str,
    bands: list[str],
    group_idxs: list[int],
    raster_format: RasterFormat,
    projection: Projection | None = None,
    bounds: PixelBounds | None = None,
    resampling: Resampling = Resampling.bilinear,
) -> list[RasterArray]:
    """Read rasters for the specified item groups.

    The default implementation loops over :meth:`read_raster`.
    """
    return [
        self.read_raster(
            layer_name,
            bands,
            raster_format,
            projection,
            bounds,
            group_idx=group_idx,
            resampling=resampling,
        )
        for group_idx in group_idxs
    ]

read_vector abstractmethod 

read_vector(layer_name: str, vector_format: VectorFormat, projection: Projection | None = None, bounds: PixelBounds | None = None, group_idx: int = 0) -> list[Feature]

Read a single item group's vector features.

Parameters:

Name Type Description Default
layer_name str

the layer name.

 required
vector_format VectorFormat

the vector format to decode with.

 required
projection Projection | None

target projection (defaults to window projection).

 None
bounds PixelBounds | None

target bounds (defaults to window bounds).

 None
group_idx int

the item group index (default 0).

 0
Source code in rslearn/dataset/window_data_storage/storage.py 
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
@abstractmethod
def read_vector(
    self,
    layer_name: str,
    vector_format: VectorFormat,
    projection: Projection | None = None,
    bounds: PixelBounds | None = None,
    group_idx: int = 0,
) -> list[Feature]:
    """Read a single item group's vector features.

    Args:
        layer_name: the layer name.
        vector_format: the vector format to decode with.
        projection: target projection (defaults to window projection).
        bounds: target bounds (defaults to window bounds).
        group_idx: the item group index (default 0).
    """

WindowDataStorageFactory

Bases: ABC

Factory that creates a :class:WindowDataStorage bound to a window.

The dataset config selects which implementation to use via :class:WindowDataStorageConfig. The dataset holds a factory and uses it to bind data storage to each loaded window.

Source code in rslearn/dataset/window_data_storage/storage.py 
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
class WindowDataStorageFactory(ABC):
    """Factory that creates a :class:`WindowDataStorage` bound to a window.

    The dataset config selects which implementation to use via
    :class:`WindowDataStorageConfig`. The dataset holds a factory and uses it
    to bind data storage to each loaded window.
    """

    @abstractmethod
    def create(self, window: Window) -> WindowDataStorage:
        """Create a WindowDataStorage bound to the given window.

        Args:
            window: the window to bind the storage to.
        """

create abstractmethod 

create(window: Window) -> WindowDataStorage

Create a WindowDataStorage bound to the given window.

Parameters:

Name Type Description Default
window Window

the window to bind the storage to.

 required
Source code in rslearn/dataset/window_data_storage/storage.py 
165
166
167
168
169
170
171
@abstractmethod
def create(self, window: Window) -> WindowDataStorage:
    """Create a WindowDataStorage bound to the given window.

    Args:
        window: the window to bind the storage to.
    """