Skip to content

Read a video's start time from its GPS clock, and tell whether creation_time marks the start or the end - #832

Open
caglarpir wants to merge 2 commits into
mapillary:mainfrom
caglarpir:fix-video-start-time
Open

caglarpir wants to merge 2 commits into
mapillary:mainfrom
caglarpir:fix-video-start-time

Conversation

@caglarpir

@caglarpir caglarpir commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

The bug

A community report (Mapillary forum + FB group): a video recorded at 4.91 fps, paired with a GPX
of the same drive, comes out with its frames on the wrong part of the route. The same file + GPX
works in Google Street View.

The reporter described it as the frame rate being misread. It isn't — the container is parsed
correctly (mdhd timescale 15712, stts 3186 × 3200 → exactly 4.91 fps, and
mp4_sample_parser reproduces every composition time to the microsecond). What is wrong is
the video's start time, and since the frames are located along the GPX by their absolute
timestamps, getting that wrong moves all of them together.

Probe.extract_stream_start_time() returned creation_time - duration. For the reported file
(creation_time 2026-08-08T08:47:32Z, duration 648.879837s) that is 08:36:43.120Z, while the
GPX starts at 08:47:32Z — the creation time, to the second. Every sampled frame came out one
full video duration early, and the workflow the README documents fails outright:

$ mapillary_tools video_process VIDEO_DIR --geotag_source gpx \
      --geotag_source_path R0024681.gpx --video_sample_distance -1 --video_sample_interval 20
...
32 image read in total
        32 (81.0 MB) MapillaryOutsideGPXTrackError
MapillaryProcessError: Failed to process 32 files.

The hard failure is the lucky case. When the track does reach back far enough — a GPX covering
a drive that the camera split across several files — the frames from every file after the first
are silently placed one file-length back along the route. On this track that is up to ~1 km,
and the images end up bunched onto road they were not shot on, which is what the report
describes.

Where it came from

The subtraction was introduced in 644701c (Feb 2019). The comment it deleted says what it was
for:

# Blackvue actually reports endtime in the created_at exif field

It was a BlackVue workaround that later got generalised to every video. But neither reading is
right for every camera, and nothing in the file says which one applies. Checked against sample
videos, by comparing the creation time with the camera's own GPS clock or with the neighbouring
clips of the same drive:

  • End of the recording: BlackVue, Viofo, Vantrue, AZDome and most other dashcams, the
    Ricoh Theta X and the Labpano PanoX V2.
  • Start of the recording: Sony, Insta360, and the reported file. GoPro also writes the
    start, but in local time (Video Timestamping Bugs with GoPro MAX #819).

The fix

Resolve the start time in this order:

  1. --video_start_time, if given (unchanged).
  2. The video's own GPS clock, which is absolute UTC and so right for both BlackVue and
    GoPro: the median of the first 5 plausible GPS timestamps, each mapped back to the video's
    time 0. Timestamps before 2010 (a GoPro writes 2000-01-01 until its first fix) or more than
    a day in the future are skipped, and the median keeps a stale first fix (seen on the PanoX
    V2, seconds older than the rest) from shifting the result.
  3. The container's creation_time, read as the start or the end of the recording by
    looking for evidence:
    • a camera known to stamp the end: BlackVue, detected by the free/gps and free/cprt
      boxes it writes whether or not it ever got a fix, and the Theta X and PanoX V2, by their
      make and model tags;
    • otherwise a date and time in the file name (20230106_150558_00422F.MP4,
      2023_0725_141524_103.MP4, …) that matches only one of creation_time and
      creation_time - duration, allowing for a whole UTC offset because cameras name files in
      local time;
    • otherwise the start, with a warning that prints the --video_start_time value to pass if
      the camera stamps the end.

A start-time source that fails (telemetry that does not parse, a malformed creation_time)
now falls through to the next one with a warning instead of failing the video.

Behaviour changes

  • Start-stamping cameras without telemetry (the reported case, Sony, Insta360): frames are
    timestamped one duration later than before, which is when they were shot.
  • End-stamping cameras without telemetry: unchanged when the camera model or the file name
    identifies them. The others now get the start reading, with the warning, where every camera
    used to get the end reading silently.
  • GoPro / CAMM with interval sampling: frames are timestamped from the GPS clock. For the
    hero8.mp4 fixture that moves the first sample from 2019_11_18_15_41_12_354 (the camera's
    local RTC, minus the duration, labelled UTC) to 2019_11_18_23_42_08_539 (real UTC, matching
    the GPS positions written into the same frames). This is Video Timestamping Bugs with GoPro MAX #819 items 2 and 4.
  • BlackVue: GPS point times were measured from the first valid fix instead of from the
    start of the recording. Until the receiver gets a fix the camera logs NMEA lines without
    positions, so every point was late by the time to first fix. They are now measured from the
    first NMEA line, which changes BlackVue point times wherever they are used, not only the
    start time. For most videos the two coincide; on 433 BlackVue sample videos, 7 took more than
    5 s to get a fix, the slowest 60 s.
  • Distance-based sampling: frames at GPS points with an absolute timestamp still take their
    time from it. The start time is resolved, in the order above, only for frames without one, so
    a video without a usable creation time no longer fails when it does not need one.

Verification

  • The reported file (make/model RICOH / RICOH THETA Z1, which is not on the
    end-stamping list, and no date in its name) resolves to 08:47:32Z, the GPX start, and gets the
    warning. That is the start time the first revision of this PR used, with which
    video_process --geotag_source gpx --video_sample_distance -1 --video_sample_interval 2
    yielded 289 geotagged frames spanning 08:47:32 → 08:58:18, every one 0.0000 m from the GPX
    position at its true video time. Before: 0 frames, all rejected as outside the track.
  • Interval sampling with a GPX, built from the camera's own track where it has one, on
    11 sample videos from 10 cameras:
    • Viofo A129 and AZDome GS63H: 119 and 57 frames geotagged, as on main. Reading every
      creation time as the start had dropped them to 1 and 0.
    • Sony FDR-X3000 with its converted GPX: 0 → 129 frames geotagged.
    • BlackVue DR900M-2CH: frames move from 5.2 s late to on time.
    • The samples whose camera clock is in local time or simply wrong fail the same way before
      and after; --video_start_time is the way out there.
  • pytest tests/unit (716 passed, 18 skipped, 1 xfailed), pytest tests/integration
    (52 passed), ruff check, ruff format --check, usort diff and mypy all pass.

Not addressed here

  • --video_geotag_source gpx still rebases the GPX onto video time 0, so a GPX covering
    several video files is injected whole into each of them. That is the documented limitation
    under "Limitations of --video_geotag_source" and is a separate change.
  • In the same code path, GPXVideoExtractor._gpx_offset syncs the GPX to the video's first GPS
    point as if that point were at video time 0, ignoring its point.time, so the GPX is off by
    however far into the video the first fix comes. This predates this PR.

… duration

A video's start time decides where its sampled frames land on a GPS track.
We derived it as creation_time - duration, which pushed every video one full
duration into the past unless its camera happened to stamp the creation time
at the end of the recording.

That subtraction came from 644701c (2019), whose removed comment says why:
"Blackvue actually reports endtime in the created_at exif field". The
workaround was later generalised to every video. ISO/IEC 14496-12 defines
creation_time as the creation time of the presentation, which for a recording
is the moment it started, and that is what cameras and ffmpeg write.

Take the start time from the video's own telemetry when it has some. That
clock is absolute UTC, so it is right for BlackVue, whose container time is
the end of the recording, and for GoPro, whose container time is in local
time (mapillary#819). Fall back to creation_time for the plain MP4s that get geotagged
from a GPX, which are exactly the videos that have no telemetry to sync
against.

Reported for a 10:49 dashcam video paired with a GPX starting at the video's
creation time: every sampled frame came out 648.88s early, so the documented
GPX workflow failed with MapillaryOutsideGPXTrackError on all of them. Where
the track does reach back far enough -- a GPX covering a recording split into
several files -- the frames were silently placed up to a kilometre off
instead. With this change all 289 sampled frames land on the track exactly.
The previous commit reads a video's start time from its telemetry and
otherwise takes creation_time as the start. That fixes the reported
dashcam, and Sony and Insta360, which also stamp the start. But it moves
every video from a camera that stamps the end one full duration late.
Those are most dashcams: Viofo, Vantrue, AZDome and others without GPS
telemetry, which people geotag from a GPX, plus the Ricoh Theta X and
the Labpano PanoX V2. With interval sampling and a GPX of their own
track, sample videos from a Viofo A129 and an AZDome GS63H went from 119
and 57 geotagged frames to 1 and 0.

Nothing in the metadata says which of the two a camera writes, so look
for evidence, in this order:

- a camera known to stamp the end: the Theta X and the PanoX V2 by their
  make and model tags, BlackVue by the boxes it writes its GPS log into,
  which it does whether or not it ever gets a fix
- a date and time in the file name that matches only one of the two,
  allowing for a whole UTC offset because cameras name files in local
  time
- otherwise the start, with a warning that prints the
  --video_start_time value to use if the camera stamps the end

Also:

- Take the median of the first 5 plausible GPS timestamps instead of the
  first one. The PanoX V2 can record a stale first fix seconds older
  than the rest, and a GPS clock that was never set (GoPro writes
  2000-01-01 until its first fix) must not become the start time. The
  hero8 fixture moves by 0.1s because the GPS5 samples of one payload
  share a timestamp.
- Fall back to the creation time when the telemetry fails to parse,
  instead of failing to sample the video.
- Measure BlackVue point times from the first NMEA line, which is the
  start of the recording, instead of from the first valid fix. Until the
  receiver gets a fix the camera logs lines without positions, so every
  point, and with the previous commit every frame sampled by interval,
  was late by the time to first fix. Across 433 BlackVue sample videos
  7 took more than 5s to get a fix, one 60s.
- Distance sampling resolves the start time only when a sampled frame
  has no absolute GPS time of its own, instead of requiring a creation
  time up front for every video.
- Ignore a malformed creation_time with a warning instead of raising.
@caglarpir caglarpir changed the title Read a video's start time from its own clock, not creation time minus duration Read a video's start time from its GPS clock, and tell whether creation_time marks the start or the end Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant