HomePhorge

Index only the first 1,000 comments on any object

Description

Index only the first 1,000 comments on any object

Summary:
Depends on D19502. Ref T13151. See PHI719. An install ended up with an object with 111,000+ comments on it because someone wrote a script to treat it like a logfile.

Although we seem to do mostly okay with this (locally, it only takes about 30s to index a similar object) we'll hit a wall somewhere (since we need to hold everything in memory), and it's hard to imagine a legitimate object with more than 1,000 comments. Just ignore comments past the first thousand.

(Conpherence threads may legitimately have more than 1,000 comments, but go through a different indexer.)

Test Plan:

  • Piped some comments into maniphest.edit in a loop to create a task with 100K comments.
  • Ran bin/search index Txxx --force to reindex it, with --trace.
    • Before: task indexed in about 30s.
    • After: script loaded comments with LIMIT 1000 and indexed in a couple seconds.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13151

Differential Revision: https://secure.phabricator.com/D19503

Details

Provenance
epriestleyAuthored on Jun 22 2018, 7:21 AM
themackabuPushed on Mar 25 2025, 8:07 PM
Parents
rPcbc71e75fae3: When queueing search index tasks, include the "objectPHID" in the task metadata
Branches
Unknown
Tags
Unknown

Event Timeline