ANDROID: mm: Fix SPF-aware fast-mremap
Fast-mremap tries to detect concerrent SPF's by checking if
vma->file_ref_count is 1 and setting it to -1 to prevent concurrent
SPF's from happening after a fast-mremap has started. This check
doesn't account for the cases where the VMA has been split/merged
after the check has happened in the SPF or fast-remap paths; which
can lead to the PMD/PUD level page table being changed from under
a concurrent SPF if certain race conditions are met.
Currently SPF takes the PTL lock at the lowest level of the
page table hierarchy (i.e. the page table page covering a 2MB extent).
Fast mremap at the PMD level only takes the PTL lock for the
page table pages (old and new) containing the PMD entries to be
updated (i.e the page table pages covering 1GB extents)
Remove the vma_ref_count trylock, and disable fast-mremap at the PUD
level if SPF is enabled.
At the PMD level take the pte ptl lock on the source address,
to avoid races with concurrent SPF.
NOTE: The additional lock, means that there are now 2 levels of
nested spinlocks. However we forego adding a new lockdep
subclass as Android doesn't enable lockdep in production.
old_ptl (pmd_lock)
new_ptl (pmd_lock)
old_pte_ptl (pte ptl lock)
Bug: 377672115
Change-Id: Ie634806115ce86a05477dfe45806487c856c3759
Signed-off-by:
Kalesh Singh <kaleshsingh@google.com>
Loading
Please register or sign in to comment