Skip to content
Snippets Groups Projects
  1. Mar 13, 2025
  2. Feb 05, 2025
    • Josip Sokcevic's avatar
      gc: Add tags to remote pack list · 8d5f0326
      Josip Sokcevic authored
      If tags are omitted from the remote pack list, they must be present in
      local pack. However, local packs don't have promisor objects, meaning
      that all blobs must be available locally, and therefore all missing
      blobs will be downloaded during rev-list phase. Git downloads those
      sequentially, by invokving fetch operation (rev-list/fetch).
      
      Instead of downloading tags' blobs, instruct Git to include all tags in
      remote rev-list operation. This change was tested with `git fsck --all`.
      
      R=yiwzhang@google.com
      
      Bug: b/392732561
      Change-Id: Id94a40aebbe4f084c952329583d559d296db1a11
      Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/451422
      
      
      Reviewed-by: default avatarYiwei Zhang <yiwzhang@google.com>
      Tested-by: default avatarJosip Sokcevic <sokcevic@chromium.org>
      Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
      v2.52
      8d5f0326
  3. Feb 04, 2025
  4. Jan 31, 2025
  5. Jan 30, 2025
  6. Jan 24, 2025
  7. Jan 16, 2025
    • Josip Sokcevic's avatar
      sync: Recover from errors during read-tree · db111d39
      Josip Sokcevic authored
      When repo is initializing a git repository, it calls `git read-tree`.
      During such operation, git is restoring workspace based on the current
      index. However, some things can go wrong: a user can run out of disk
      space, or, in case of partial clone, user may no longer reach the remote
      host. That will leave affected repository in a bad state with partially
      checked out workspace. The follow up repo sync won't try to fix such
      state.
      
      This change removes .git symlink, which will force the next `repo sync`
      to redo Git repository setup.
      
      Bug: b/363171216
      Bug: b/390161127
      Change-Id: I57db4b6cae0ef21826dc7cede4d3bf02cfc3d955
      Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/447801
      
      
      Reviewed-by: default avatarScott Lee <ddoman@google.com>
      Tested-by: default avatarJosip Sokcevic <sokcevic@chromium.org>
      Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
      v2.51
      db111d39
  8. Jan 14, 2025
  9. Jan 08, 2025
  10. Dec 18, 2024
  11. Dec 16, 2024
  12. Dec 11, 2024
  13. Dec 03, 2024
  14. Nov 26, 2024
  15. Nov 22, 2024
  16. Nov 18, 2024
  17. Nov 06, 2024
  18. Oct 31, 2024
  19. Oct 30, 2024
  20. Oct 28, 2024
  21. Oct 24, 2024
  22. Oct 23, 2024
    • Kuang-che Wu's avatar
      subcmds: reduce multiprocessing serialization overhead · 8da4861b
      Kuang-che Wu authored
      Follow the same approach as 39ffd997 to reduce serialization overhead.
      
      Below benchmarks are tested with 2.7k projects on my workstation
      (warm cache). git tracing is disabled for benchmark.
      
      (seconds)              | v2.48 | v2.48 | this CL | this CL
      	               |       |  -j32 |         |    -j32
      -----------------------------------------------------------
      with clean tree state:
      branches (none)        |   5.6 |   5.9 |    1.0  |    0.9
      status (clean)         |  21.3 |   9.4 |   19.4  |    4.7
      diff (none)            |   7.6 |   7.2 |    5.7  |    2.2
      prune (none)           |   5.7 |   6.1 |    1.3  |    1.2
      abandon (none)         |  19.4 |  18.6 |    0.9  |    0.8
      upload (none)          |  19.7 |  18.7 |    0.9  |    0.8
      forall -c true         |   7.5 |   7.6 |    0.6  |    0.6
      forall -c "git log -1" |  11.3 |  11.1 |    0.6  |    0.6
      
      with branches:
      start BRANCH --all     |  21.9 |  20.3 |   13.6  |    2.6
      checkout BRANCH        |  29.1 |  27.8 |    1.1  |    1.0
      branches (2)           |  28.0 |  28.6 |    1.5  |    1.3
      abandon BRANCH         |  29.2 |  27.5 |    9.7  |    2.2
      
      Bug: b/371638995
      Change-Id: I53989a3d1e43063587b3f52f852b1c2c56b49412
      Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440221
      
      
      Reviewed-by: default avatarJosip Sokcevic <sokcevic@google.com>
      Tested-by: default avatarKuang-che Wu <kcwu@google.com>
      Commit-Queue: Kuang-che Wu <kcwu@google.com>
      8da4861b
    • Kuang-che Wu's avatar
      sync: reduce multiprocessing serialization overhead · 39ffd997
      Kuang-che Wu authored
      Background:
       - Manifest object is large (for projects like Android) in terms of
         serialization cost and size (more than 1mb).
       - Lots of Project objects usually share only a few manifest objects.
      
      Before this CL, Project objects were passed to workers via function
      parameters. Function parameters are pickled separately (in chunk). In
      other words, manifests are serialized again and again. The major
      serialization overhead of repo sync was
        O(manifest_size * projects / chunksize)
      
      This CL uses following tricks to reduce serialization overhead.
       - All projects are pickled in one invocation. Because Project objects
         share manifests, pickle library remembers which objects are already
         seen and avoid the serialization cost.
       - Pass the Project objects to workers at worker intialization time.
         And pass project index as function parameters instead. The number of
         workers is much smaller than the number of projects.
       - Worker init state are shared on Linux (fork based). So it requires
         zero serialization for Project objects.
      
      On Linux (fork based), the serialization overhead is
        O(projects)  --- one int per project
      On Windows (spawn based), the serialization overhead is
        O(manifest_size * min(workers, projects))
      
      Moreover, use chunksize=1 to avoid the chance that some workers are idle
      while other workers still have more than one job in their chunk queue.
      
      Using 2.7k projects as the baseline, originally "repo sync" no-op
      sync takes 31s for fetch and 25s for checkout on my Linux workstation.
      With this CL, it takes 12s for fetch and 1s for checkout.
      
      Bug: b/371638995
      Change-Id: Ifa22072ea54eacb4a5c525c050d84de371e87caa
      Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439921
      
      
      Tested-by: default avatarKuang-che Wu <kcwu@google.com>
      Reviewed-by: default avatarJosip Sokcevic <sokcevic@google.com>
      Commit-Queue: Kuang-che Wu <kcwu@google.com>
      39ffd997
  23. Oct 18, 2024
  24. Oct 07, 2024
  25. Oct 03, 2024
  26. Sep 26, 2024
Loading