- 22 3月, 2022 40 次提交
-
-
由 Matthew Wilcox (Oracle) 提交于
These functions are page cache functionality and don't need to be declared in fs.h. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NMiaohe Lin <linmiaohe@huawei.com>
-
由 Matthew Wilcox (Oracle) 提交于
We can save a function call by combining these two functions, which are identical except for the return value. Also move the prototype to mm/internal.h. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NMiaohe Lin <linmiaohe@huawei.com>
-
由 Matthew Wilcox (Oracle) 提交于
This function has one caller which already has a reference to the page, so we don't need to use get_page_unless_zero(). Also move the prototype to mm/internal.h. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NMiaohe Lin <linmiaohe@huawei.com>
-
由 Matthew Wilcox (Oracle) 提交于
Now we can call mapping_evict_folio() instead of invalidate_inode_page() and save a few calls to compound_head(). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NMiaohe Lin <linmiaohe@huawei.com>
-
由 Matthew Wilcox (Oracle) 提交于
Some of the callers already have the address_space and can avoid calling folio_mapping() and checking if the folio was already truncated. Also add kernel-doc and fix the return type (in case we ever support folios larger than 4TB). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NMiaohe Lin <linmiaohe@huawei.com>
-
由 Matthew Wilcox (Oracle) 提交于
Add kernel-doc and return the number of pages removed in order to get the statistics right in __invalidate_mapping_pages(). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NMiaohe Lin <linmiaohe@huawei.com>
-
由 Matthew Wilcox (Oracle) 提交于
folio_mapped() is expensive because it has to check each page's mapcount field. A cheaper check is whether there are any extra references to the page, other than the one we own, one from the page private data and the ones held by the page cache. The call to remove_mapping() will fail in any case if it cannot freeze the refcount, but failing here avoids cycling the i_pages spinlock. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NMiaohe Lin <linmiaohe@huawei.com>
-
由 Matthew Wilcox (Oracle) 提交于
This saves a number of calls to compound_head(). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NMiaohe Lin <linmiaohe@huawei.com>
-
由 Matthew Wilcox (Oracle) 提交于
invalidate_inode_page() is the only caller of invalidate_complete_page() and inlining it reveals that the first check is unnecessary (because we hold the page locked, and we just retrieved the mapping from the page). Actually, it does make a difference, in that tail pages no longer fail at this check, so it's now possible to remove a tail page from a mapping. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NChristoph Hellwig <hch@lst.de>
-
由 Matthew Wilcox (Oracle) 提交于
This saves a lot of calls to compound_head(). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NMiaohe Lin <linmiaohe@huawei.com>
-
由 Matthew Wilcox (Oracle) 提交于
This removes a few hidden calls to compound_head(). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de>
-
由 Matthew Wilcox (Oracle) 提交于
Add a putback_lru_page() wrapper. Removes a couple of compound_head() calls. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de>
-
由 Matthew Wilcox (Oracle) 提交于
Since page->lru occupies the same bytes as compound_head, any page on the LRU list must be a folio. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org>
-
由 Matthew Wilcox (Oracle) 提交于
This removes an assumption that THPs are the only kind of compound pages and removes a couple of hidden calls to compound_head. It also documents that you can't pass a tail page to mem_cgroup_swapout(). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de>
-
由 Matthew Wilcox (Oracle) 提交于
This removes an assumption that THPs are the only kind of compound pages and removes a few hidden calls to compound_head(). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de>
-
由 Matthew Wilcox (Oracle) 提交于
Switch from head pages to folios. This removes an assumption that THPs are the only way to have a high-order page. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
folio_is_zone_device() is equivalent to is_zone_device_page(), folio_is_device_private() is equivalent to is_device_private_page(), and folio_is_pinnable() is equivalent to is_pinnable_page(). All of these tests return the same result for every page in the folio, so we can just pass the head page of the folio to the page variant of the function. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org>
-
由 Matthew Wilcox (Oracle) 提交于
Add isolate_lru_page() as a wrapper around isolate_lru_folio(). TestClearPageLRU() would have always failed on a tail page, so returning -EBUSY is the same behaviour. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Convert the only caller to work on folios instead of pages. This removes the last caller of put_compound_head(), so delete it. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Convert both callers to work on folios instead of pages. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Use the new folio-based APIs. This was the last user of try_grab_compound_head(), so remove it. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Use the new folio-based APIs. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Use the new folio-based APIs. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
There should be little to no effect from this patch; just removing uses of some old APIs. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
We still call try_grab_folio() once per PTE; a future patch could optimise to just adjust the reference count for each page within the folio. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
follow_hugetlb_page() only cares about success or failure, so it doesn't need to know the type of the returned pointer, only whether it's NULL or not. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Convert put_compound_head() to gup_put_folio() and hpage_pincount_sub() to folio_pincount_sub(). This removes the last call to put_page_refs(), so delete it. Add a temporary put_compound_head() wrapper which will be deleted by the end of this series. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
These wrappers have no more callers, so delete them. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Hoist the folio conversion and the folio_ref_count() check to the top of the function instead of using the one buried in try_get_page(). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NChristoph Hellwig <hch@lst.de>
-
由 Matthew Wilcox (Oracle) 提交于
Convert try_get_compound_head() into try_get_folio() and convert try_grab_compound_head() into try_grab_folio(). Add a temporary try_grab_compound_head() wrapper around try_grab_folio() to let us convert callers individually. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Replace three calls to compound_head() with one. This removes the last user of compound_pincount(), so remove that helper too. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
This is the folio equivalent of compound_pincount_ptr(). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Move compound_pincount from the third page to the second page, which means it's available for all compound pages. That lets us delete hpage_pincount_available(). On 32-bit systems, there isn't enough space for both compound_pincount and compound_nr in the second page (it would collide with page->private, which is in use for pages in the swap cache), so revert the optimisation of storing both compound_order and compound_nr on 32-bit systems. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Move the assertion (and correct it to be a cheaper variant), and inline the atomic_sub() operation. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
It's clearer to call atomic_add() in the callers; the assertions clearly can't fire there because they're part of the condition for calling atomic_add(). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NChristoph Hellwig <hch@lst.de>
-
由 Matthew Wilcox (Oracle) 提交于
If we hit the page split race, the current code returns NULL which will presumably trigger a retry under the mmap_lock. This isn't necessary; we can just retry the compound_head() lookup. This is a very minor optimisation of an unlikely path, but conceptually it matches (eg) the page cache RCU-protected lookup. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
This assumption needs the inverse of nth_page(), which is temporarily named page_nth() until it's renamed later in this series. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Several functions in gup.c assume that a compound page has virtually contiguous page structs. This isn't true for SPARSEMEM configs unless SPARSEMEM_VMEMMAP is also set. Fix them by using nth_page() instead of plain pointer arithmetic. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
Return the head page instead of storing it to a passed parameter. Reorder the arguments to match the calling function's arguments. Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-
由 Matthew Wilcox (Oracle) 提交于
By definition, a compound page has an order >= 1, so the second half of the test was redundant. Also, this cannot be a tail page since it's the result of calling compound_head(), so use PageHead() instead of PageCompound(). Signed-off-by: NMatthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: NChristoph Hellwig <hch@lst.de> Reviewed-by: NJohn Hubbard <jhubbard@nvidia.com> Reviewed-by: NJason Gunthorpe <jgg@nvidia.com> Reviewed-by: NWilliam Kucharski <william.kucharski@oracle.com>
-