Skip to content

open_geotiff(gpu=True, chunks=...) reads and parses a local file twice at graph build #3373

@brendancol

Description

@brendancol

Problem

_read_geotiff_gpu_chunked in xrspatial/geotiff/_backends/gpu.py reads a local source file off disk twice and parses its IFDs twice, both at dask-graph-build time.

The per-tile byte-count safety-cap block (around lines 1367-1392) opens a _FileSource(src_path), calls read_all(), then parse_header(raw) and parse_all_ifds(raw, header). It uses the parsed IFDs only for the cap loop and discards them. The GDS qualification block right after (around lines 1398-1404) opens the same file again, calls read_all() again, and runs parse_header + parse_all_ifds a second time.

So every open_geotiff(gpu=True, chunks=...) on a local file pays one extra full-file read plus one extra IFD parse. IFD parsing is O(tile_count), so on a COG with 10k+ tiles the second parse costs real time. This is graph-construction overhead paid once per open call, not a per-tile or per-chunk decode cost.

It was recorded as a deferred low-severity item in an earlier performance sweep pass. Raising it now because the cost scales with tile count on large COGs.

Proposed fix

Read the file once and parse the header and IFDs once, before the cap loop. Run the per-tile cap check on that single parse, then reuse the same raw / header / ifds for the GDS qualification probe (select_overview_ifd, extract_geo_info_with_overview_inheritance, _gds_chunk_path_available). No second read_all / parse_header / parse_all_ifds.

Constraints to preserve

  • The per-tile cap check must still raise ValueError (the denial-of-service guard) before any GDS qualification work runs.
  • The qualification block currently lets any exception fall through to the CPU path (_read_geotiff_dask); that fall-through must stay. A metadata-parse failure must not raise out of this function, it must route to the CPU path exactly as today. A genuine over-cap tile must still raise.
  • The local-file-only guard (isinstance(src_path, str) and not http/https) stays; the HTTP branch is separate and untouched.
  • This block was hardened for a free-threaded (no-GIL) concurrency race; the _FileSource context-manager usage and scoping must not be weakened.

Scope

Graph-build path only: one read and one parse saved per open call. Not a decode-throughput change. geotiff is not in the performance-labeler benchmarked set, so no ASV benchmark runs on this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestgeotiffGeoTIFF modulegpuCuPy / CUDA GPU supportperformancePR touches performance-sensitive codesweep-performanceFound by /sweep-performance

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions