HomePhorge

As Harbormaster logs are processed, build a sparse map of byte offsets to line…

Description

As Harbormaster logs are processed, build a sparse map of byte offsets to line numbers

Summary:
Depends on D19138. Ref T13088. When we want to read the last part of a logfile and show accurate line numbers, we need to be able to get from byte offsets to line numbers somehow.

Our fundamental unit must remain byte offsets, because a test can emit an arbitrarily long line, and we should accommodate it cleanly if a test emits 2GB of the letter "A".

To support going from byte offsets to line numbers, compute a map with periodic line markers throughout the offsets of the file. From here, we can figure out the line numbers for arbitrary positions in the file with only a constant amount of work.

Test Plan: Added unit tests; ran unit tests.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13088

Differential Revision: https://secure.phabricator.com/D19139

Details

Provenance
epriestleyAuthored on Feb 23 2018, 7:10 AM
themackabuPushed on Mar 25 2025, 8:07 PM
Parents
rPd6311044bba6: Store the Harbormaster log chunk format on the log record
Branches
Unknown
Tags
Unknown

Event Timeline