Conversation
… duration A video's start time decides where its sampled frames land on a GPS track. We derived it as creation_time - duration, which pushed every video one full duration into the past unless its camera happened to stamp the creation time at the end of the recording. That subtraction came from 644701c (2019), whose removed comment says why: "Blackvue actually reports endtime in the created_at exif field". The workaround was later generalised to every video. ISO/IEC 14496-12 defines creation_time as the creation time of the presentation, which for a recording is the moment it started, and that is what cameras and ffmpeg write. Take the start time from the video's own telemetry when it has some. That clock is absolute UTC, so it is right for BlackVue, whose container time is the end of the recording, and for GoPro, whose container time is in local time (mapillary#819). Fall back to creation_time for the plain MP4s that get geotagged from a GPX, which are exactly the videos that have no telemetry to sync against. Reported for a 10:49 dashcam video paired with a GPX starting at the video's creation time: every sampled frame came out 648.88s early, so the documented GPX workflow failed with MapillaryOutsideGPXTrackError on all of them. Where the track does reach back far enough -- a GPX covering a recording split into several files -- the frames were silently placed up to a kilometre off instead. With this change all 289 sampled frames land on the track exactly.
The previous commit reads a video's start time from its telemetry and otherwise takes creation_time as the start. That fixes the reported dashcam, and Sony and Insta360, which also stamp the start. But it moves every video from a camera that stamps the end one full duration late. Those are most dashcams: Viofo, Vantrue, AZDome and others without GPS telemetry, which people geotag from a GPX, plus the Ricoh Theta X and the Labpano PanoX V2. With interval sampling and a GPX of their own track, sample videos from a Viofo A129 and an AZDome GS63H went from 119 and 57 geotagged frames to 1 and 0. Nothing in the metadata says which of the two a camera writes, so look for evidence, in this order: - a camera known to stamp the end: the Theta X and the PanoX V2 by their make and model tags, BlackVue by the boxes it writes its GPS log into, which it does whether or not it ever gets a fix - a date and time in the file name that matches only one of the two, allowing for a whole UTC offset because cameras name files in local time - otherwise the start, with a warning that prints the --video_start_time value to use if the camera stamps the end Also: - Take the median of the first 5 plausible GPS timestamps instead of the first one. The PanoX V2 can record a stale first fix seconds older than the rest, and a GPS clock that was never set (GoPro writes 2000-01-01 until its first fix) must not become the start time. The hero8 fixture moves by 0.1s because the GPS5 samples of one payload share a timestamp. - Fall back to the creation time when the telemetry fails to parse, instead of failing to sample the video. - Measure BlackVue point times from the first NMEA line, which is the start of the recording, instead of from the first valid fix. Until the receiver gets a fix the camera logs lines without positions, so every point, and with the previous commit every frame sampled by interval, was late by the time to first fix. Across 433 BlackVue sample videos 7 took more than 5s to get a fix, one 60s. - Distance sampling resolves the start time only when a sampled frame has no absolute GPS time of its own, instead of requiring a creation time up front for every video. - Ignore a malformed creation_time with a warning instead of raising.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
A community report (Mapillary forum + FB group): a video recorded at 4.91 fps, paired with a GPX
of the same drive, comes out with its frames on the wrong part of the route. The same file + GPX
works in Google Street View.
The reporter described it as the frame rate being misread. It isn't — the container is parsed
correctly (
mdhdtimescale 15712,stts3186 × 3200 → exactly 4.91 fps, andmp4_sample_parserreproduces every composition time to the microsecond). What is wrong isthe video's start time, and since the frames are located along the GPX by their absolute
timestamps, getting that wrong moves all of them together.
Probe.extract_stream_start_time()returnedcreation_time - duration. For the reported file(
creation_time2026-08-08T08:47:32Z, duration 648.879837s) that is 08:36:43.120Z, while theGPX starts at 08:47:32Z — the creation time, to the second. Every sampled frame came out one
full video duration early, and the workflow the README documents fails outright:
The hard failure is the lucky case. When the track does reach back far enough — a GPX covering
a drive that the camera split across several files — the frames from every file after the first
are silently placed one file-length back along the route. On this track that is up to ~1 km,
and the images end up bunched onto road they were not shot on, which is what the report
describes.
Where it came from
The subtraction was introduced in 644701c (Feb 2019). The comment it deleted says what it was
for:
# Blackvue actually reports endtime in the created_at exif fieldIt was a BlackVue workaround that later got generalised to every video. But neither reading is
right for every camera, and nothing in the file says which one applies. Checked against sample
videos, by comparing the creation time with the camera's own GPS clock or with the neighbouring
clips of the same drive:
Ricoh Theta X and the Labpano PanoX V2.
start, but in local time (Video Timestamping Bugs with GoPro MAX #819).
The fix
Resolve the start time in this order:
--video_start_time, if given (unchanged).GoPro: the median of the first 5 plausible GPS timestamps, each mapped back to the video's
time 0. Timestamps before 2010 (a GoPro writes 2000-01-01 until its first fix) or more than
a day in the future are skipped, and the median keeps a stale first fix (seen on the PanoX
V2, seconds older than the rest) from shifting the result.
creation_time, read as the start or the end of the recording bylooking for evidence:
free/gpsandfree/cprtboxes it writes whether or not it ever got a fix, and the Theta X and PanoX V2, by their
make and model tags;
20230106_150558_00422F.MP4,2023_0725_141524_103.MP4, …) that matches only one ofcreation_timeandcreation_time - duration, allowing for a whole UTC offset because cameras name files inlocal time;
--video_start_timevalue to pass ifthe camera stamps the end.
A start-time source that fails (telemetry that does not parse, a malformed
creation_time)now falls through to the next one with a warning instead of failing the video.
Behaviour changes
timestamped one duration later than before, which is when they were shot.
identifies them. The others now get the start reading, with the warning, where every camera
used to get the end reading silently.
hero8.mp4fixture that moves the first sample from2019_11_18_15_41_12_354(the camera'slocal RTC, minus the duration, labelled UTC) to
2019_11_18_23_42_08_539(real UTC, matchingthe GPS positions written into the same frames). This is Video Timestamping Bugs with GoPro MAX #819 items 2 and 4.
start of the recording. Until the receiver gets a fix the camera logs NMEA lines without
positions, so every point was late by the time to first fix. They are now measured from the
first NMEA line, which changes BlackVue point times wherever they are used, not only the
start time. For most videos the two coincide; on 433 BlackVue sample videos, 7 took more than
5 s to get a fix, the slowest 60 s.
time from it. The start time is resolved, in the order above, only for frames without one, so
a video without a usable creation time no longer fails when it does not need one.
Verification
RICOH/RICOH THETA Z1, which is not on theend-stamping list, and no date in its name) resolves to 08:47:32Z, the GPX start, and gets the
warning. That is the start time the first revision of this PR used, with which
video_process --geotag_source gpx --video_sample_distance -1 --video_sample_interval 2yielded 289 geotagged frames spanning 08:47:32 → 08:58:18, every one 0.0000 m from the GPX
position at its true video time. Before: 0 frames, all rejected as outside the track.
11 sample videos from 10 cameras:
main. Reading everycreation time as the start had dropped them to 1 and 0.
and after;
--video_start_timeis the way out there.pytest tests/unit(716 passed, 18 skipped, 1 xfailed),pytest tests/integration(52 passed),
ruff check,ruff format --check,usort diffandmypyall pass.Not addressed here
--video_geotag_source gpxstill rebases the GPX onto video time 0, so a GPX coveringseveral video files is injected whole into each of them. That is the documented limitation
under "Limitations of
--video_geotag_source" and is a separate change.GPXVideoExtractor._gpx_offsetsyncs the GPX to the video's first GPSpoint as if that point were at video time 0, ignoring its
point.time, so the GPX is off byhowever far into the video the first fix comes. This predates this PR.