• A
    mm/memunmap: don't access uninitialized memmap in memunmap_pages() · 77e080e7
    Aneesh Kumar K.V 提交于
    Patch series "mm/memory_hotplug: Shrink zones before removing memory",
    v6.
    
    This series fixes the access of uninitialized memmaps when shrinking
    zones/nodes and when removing memory.  Also, it contains all fixes for
    crashes that can be triggered when removing certain namespace using
    memunmap_pages() - ZONE_DEVICE, reported by Aneesh.
    
    We stop trying to shrink ZONE_DEVICE, as it's buggy, fixing it would be
    more involved (we don't have SECTION_IS_ONLINE as an indicator), and
    shrinking is only of limited use (set_zone_contiguous() cannot detect
    the ZONE_DEVICE as contiguous).
    
    We continue shrinking !ZONE_DEVICE zones, however, I reduced the amount
    of code to a minimum.  Shrinking is especially necessary to keep
    zone->contiguous set where possible, especially, on memory unplug of
    DIMMs at zone boundaries.
    
    --------------------------------------------------------------------------
    
    Zones are now properly shrunk when offlining memory blocks or when
    onlining failed.  This allows to properly shrink zones on memory unplug
    even if the separate memory blocks of a DIMM were onlined to different
    zones or re-onlined to a different zone after offlining.
    
    Example:
    
      :/# cat /proc/zoneinfo
      Node 1, zone  Movable
              spanned  0
              present  0
              managed  0
      :/# echo "online_movable" > /sys/devices/system/memory/memory41/state
      :/# echo "online_movable" > /sys/devices/system/memory/memory43/state
      :/# cat /proc/zoneinfo
      Node 1, zone  Movable
              spanned  98304
              present  65536
              managed  65536
      :/# echo 0 > /sys/devices/system/memory/memory43/online
      :/# cat /proc/zoneinfo
      Node 1, zone  Movable
              spanned  32768
              present  32768
              managed  32768
      :/# echo 0 > /sys/devices/system/memory/memory41/online
      :/# cat /proc/zoneinfo
      Node 1, zone  Movable
              spanned  0
              present  0
              managed  0
    
    This patch (of 10):
    
    With an altmap, the memmap falling into the reserved altmap space are not
    initialized and, therefore, contain a garbage NID and a garbage zone.
    Make sure to read the NID/zone from a memmap that was initialized.
    
    This fixes a kernel crash that is observed when destroying a namespace:
    
      kernel BUG at include/linux/mm.h:1107!
      cpu 0x1: Vector: 700 (Program Check) at [c000000274087890]
          pc: c0000000004b9728: memunmap_pages+0x238/0x340
          lr: c0000000004b9724: memunmap_pages+0x234/0x340
      ...
          pid   = 3669, comm = ndctl
      kernel BUG at include/linux/mm.h:1107!
        devm_action_release+0x30/0x50
        release_nodes+0x268/0x2d0
        device_release_driver_internal+0x174/0x240
        unbind_store+0x13c/0x190
        drv_attr_store+0x44/0x60
        sysfs_kf_write+0x70/0xa0
        kernfs_fop_write+0x1ac/0x290
        __vfs_write+0x3c/0x70
        vfs_write+0xe4/0x200
        ksys_write+0x7c/0x140
        system_call+0x5c/0x68
    
    The "page_zone(pfn_to_page(pfn)" was introduced by 69324b8f ("mm,
    devm_memremap_pages: add MEMORY_DEVICE_PRIVATE support"), however, I
    think we will never have driver reserved memory with
    MEMORY_DEVICE_PRIVATE (no altmap AFAIKS).
    
    [david@redhat.com: minimze code changes, rephrase description]
    Link: http://lkml.kernel.org/r/20191006085646.5768-2-david@redhat.com
    Fixes: 2c2a5af6 ("mm, memory_hotplug: add nid parameter to arch_remove_memory")
    Signed-off-by: NAneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
    Signed-off-by: NDavid Hildenbrand <david@redhat.com>
    Cc: Dan Williams <dan.j.williams@intel.com>
    Cc: Jason Gunthorpe <jgg@ziepe.ca>
    Cc: Logan Gunthorpe <logang@deltatee.com>
    Cc: Ira Weiny <ira.weiny@intel.com>
    Cc: Damian Tometzki <damian.tometzki@gmail.com>
    Cc: Alexander Duyck <alexander.h.duyck@linux.intel.com>
    Cc: Alexander Potapenko <glider@google.com>
    Cc: Andy Lutomirski <luto@kernel.org>
    Cc: Anshuman Khandual <anshuman.khandual@arm.com>
    Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
    Cc: Borislav Petkov <bp@alien8.de>
    Cc: Catalin Marinas <catalin.marinas@arm.com>
    Cc: Christian Borntraeger <borntraeger@de.ibm.com>
    Cc: Christophe Leroy <christophe.leroy@c-s.fr>
    Cc: Dave Hansen <dave.hansen@linux.intel.com>
    Cc: Fenghua Yu <fenghua.yu@intel.com>
    Cc: Gerald Schaefer <gerald.schaefer@de.ibm.com>
    Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
    Cc: Halil Pasic <pasic@linux.ibm.com>
    Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
    Cc: "H. Peter Anvin" <hpa@zytor.com>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Jun Yao <yaojun8558363@gmail.com>
    Cc: Mark Rutland <mark.rutland@arm.com>
    Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
    Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>
    Cc: Mel Gorman <mgorman@techsingularity.net>
    Cc: Michael Ellerman <mpe@ellerman.id.au>
    Cc: Michal Hocko <mhocko@suse.com>
    Cc: Mike Rapoport <rppt@linux.ibm.com>
    Cc: Oscar Salvador <osalvador@suse.de>
    Cc: Pankaj Gupta <pagupta@redhat.com>
    Cc: Paul Mackerras <paulus@samba.org>
    Cc: Pavel Tatashin <pasha.tatashin@soleen.com>
    Cc: Pavel Tatashin <pavel.tatashin@microsoft.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Qian Cai <cai@lca.pw>
    Cc: Rich Felker <dalias@libc.org>
    Cc: Robin Murphy <robin.murphy@arm.com>
    Cc: Steve Capper <steve.capper@arm.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Tom Lendacky <thomas.lendacky@amd.com>
    Cc: Tony Luck <tony.luck@intel.com>
    Cc: Vasily Gorbik <gor@linux.ibm.com>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Wei Yang <richard.weiyang@gmail.com>
    Cc: Wei Yang <richardw.yang@linux.intel.com>
    Cc: Will Deacon <will@kernel.org>
    Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
    Cc: Yu Zhao <yuzhao@google.com>
    Cc: <stable@vger.kernel.org>	[5.0+]
    Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
    77e080e7
memremap.c 12.2 KB
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457
/* SPDX-License-Identifier: GPL-2.0 */
/* Copyright(c) 2015 Intel Corporation. All rights reserved. */
#include <linux/device.h>
#include <linux/io.h>
#include <linux/kasan.h>
#include <linux/memory_hotplug.h>
#include <linux/mm.h>
#include <linux/pfn_t.h>
#include <linux/swap.h>
#include <linux/swapops.h>
#include <linux/types.h>
#include <linux/wait_bit.h>
#include <linux/xarray.h>

static DEFINE_XARRAY(pgmap_array);

#ifdef CONFIG_DEV_PAGEMAP_OPS
DEFINE_STATIC_KEY_FALSE(devmap_managed_key);
EXPORT_SYMBOL(devmap_managed_key);
static atomic_t devmap_managed_enable;

static void devmap_managed_enable_put(void)
{
	if (atomic_dec_and_test(&devmap_managed_enable))
		static_branch_disable(&devmap_managed_key);
}

static int devmap_managed_enable_get(struct dev_pagemap *pgmap)
{
	if (!pgmap->ops || !pgmap->ops->page_free) {
		WARN(1, "Missing page_free method\n");
		return -EINVAL;
	}

	if (atomic_inc_return(&devmap_managed_enable) == 1)
		static_branch_enable(&devmap_managed_key);
	return 0;
}
#else
static int devmap_managed_enable_get(struct dev_pagemap *pgmap)
{
	return -EINVAL;
}
static void devmap_managed_enable_put(void)
{
}
#endif /* CONFIG_DEV_PAGEMAP_OPS */

static void pgmap_array_delete(struct resource *res)
{
	xa_store_range(&pgmap_array, PHYS_PFN(res->start), PHYS_PFN(res->end),
			NULL, GFP_KERNEL);
	synchronize_rcu();
}

static unsigned long pfn_first(struct dev_pagemap *pgmap)
{
	return PHYS_PFN(pgmap->res.start) +
		vmem_altmap_offset(pgmap_altmap(pgmap));
}

static unsigned long pfn_end(struct dev_pagemap *pgmap)
{
	const struct resource *res = &pgmap->res;

	return (res->start + resource_size(res)) >> PAGE_SHIFT;
}

static unsigned long pfn_next(unsigned long pfn)
{
	if (pfn % 1024 == 0)
		cond_resched();
	return pfn + 1;
}

#define for_each_device_pfn(pfn, map) \
	for (pfn = pfn_first(map); pfn < pfn_end(map); pfn = pfn_next(pfn))

static void dev_pagemap_kill(struct dev_pagemap *pgmap)
{
	if (pgmap->ops && pgmap->ops->kill)
		pgmap->ops->kill(pgmap);
	else
		percpu_ref_kill(pgmap->ref);
}

static void dev_pagemap_cleanup(struct dev_pagemap *pgmap)
{
	if (pgmap->ops && pgmap->ops->cleanup) {
		pgmap->ops->cleanup(pgmap);
	} else {
		wait_for_completion(&pgmap->done);
		percpu_ref_exit(pgmap->ref);
	}
	/*
	 * Undo the pgmap ref assignment for the internal case as the
	 * caller may re-enable the same pgmap.
	 */
	if (pgmap->ref == &pgmap->internal_ref)
		pgmap->ref = NULL;
}

void memunmap_pages(struct dev_pagemap *pgmap)
{
	struct resource *res = &pgmap->res;
	struct page *first_page;
	unsigned long pfn;
	int nid;

	dev_pagemap_kill(pgmap);
	for_each_device_pfn(pfn, pgmap)
		put_page(pfn_to_page(pfn));
	dev_pagemap_cleanup(pgmap);

	/* make sure to access a memmap that was actually initialized */
	first_page = pfn_to_page(pfn_first(pgmap));

	/* pages are dead and unused, undo the arch mapping */
	nid = page_to_nid(first_page);

	mem_hotplug_begin();
	if (pgmap->type == MEMORY_DEVICE_PRIVATE) {
		__remove_pages(page_zone(first_page), PHYS_PFN(res->start),
			       PHYS_PFN(resource_size(res)), NULL);
	} else {
		arch_remove_memory(nid, res->start, resource_size(res),
				pgmap_altmap(pgmap));
		kasan_remove_zero_shadow(__va(res->start), resource_size(res));
	}
	mem_hotplug_done();

	untrack_pfn(NULL, PHYS_PFN(res->start), resource_size(res));
	pgmap_array_delete(res);
	WARN_ONCE(pgmap->altmap.alloc, "failed to free all reserved pages\n");
	devmap_managed_enable_put();
}
EXPORT_SYMBOL_GPL(memunmap_pages);

static void devm_memremap_pages_release(void *data)
{
	memunmap_pages(data);
}

static void dev_pagemap_percpu_release(struct percpu_ref *ref)
{
	struct dev_pagemap *pgmap =
		container_of(ref, struct dev_pagemap, internal_ref);

	complete(&pgmap->done);
}

/*
 * Not device managed version of dev_memremap_pages, undone by
 * memunmap_pages().  Please use dev_memremap_pages if you have a struct
 * device available.
 */
void *memremap_pages(struct dev_pagemap *pgmap, int nid)
{
	struct resource *res = &pgmap->res;
	struct dev_pagemap *conflict_pgmap;
	struct mhp_restrictions restrictions = {
		/*
		 * We do not want any optional features only our own memmap
		 */
		.altmap = pgmap_altmap(pgmap),
	};
	pgprot_t pgprot = PAGE_KERNEL;
	int error, is_ram;
	bool need_devmap_managed = true;

	switch (pgmap->type) {
	case MEMORY_DEVICE_PRIVATE:
		if (!IS_ENABLED(CONFIG_DEVICE_PRIVATE)) {
			WARN(1, "Device private memory not supported\n");
			return ERR_PTR(-EINVAL);
		}
		if (!pgmap->ops || !pgmap->ops->migrate_to_ram) {
			WARN(1, "Missing migrate_to_ram method\n");
			return ERR_PTR(-EINVAL);
		}
		break;
	case MEMORY_DEVICE_FS_DAX:
		if (!IS_ENABLED(CONFIG_ZONE_DEVICE) ||
		    IS_ENABLED(CONFIG_FS_DAX_LIMITED)) {
			WARN(1, "File system DAX not supported\n");
			return ERR_PTR(-EINVAL);
		}
		break;
	case MEMORY_DEVICE_DEVDAX:
	case MEMORY_DEVICE_PCI_P2PDMA:
		need_devmap_managed = false;
		break;
	default:
		WARN(1, "Invalid pgmap type %d\n", pgmap->type);
		break;
	}

	if (!pgmap->ref) {
		if (pgmap->ops && (pgmap->ops->kill || pgmap->ops->cleanup))
			return ERR_PTR(-EINVAL);

		init_completion(&pgmap->done);
		error = percpu_ref_init(&pgmap->internal_ref,
				dev_pagemap_percpu_release, 0, GFP_KERNEL);
		if (error)
			return ERR_PTR(error);
		pgmap->ref = &pgmap->internal_ref;
	} else {
		if (!pgmap->ops || !pgmap->ops->kill || !pgmap->ops->cleanup) {
			WARN(1, "Missing reference count teardown definition\n");
			return ERR_PTR(-EINVAL);
		}
	}

	if (need_devmap_managed) {
		error = devmap_managed_enable_get(pgmap);
		if (error)
			return ERR_PTR(error);
	}

	conflict_pgmap = get_dev_pagemap(PHYS_PFN(res->start), NULL);
	if (conflict_pgmap) {
		WARN(1, "Conflicting mapping in same section\n");
		put_dev_pagemap(conflict_pgmap);
		error = -ENOMEM;
		goto err_array;
	}

	conflict_pgmap = get_dev_pagemap(PHYS_PFN(res->end), NULL);
	if (conflict_pgmap) {
		WARN(1, "Conflicting mapping in same section\n");
		put_dev_pagemap(conflict_pgmap);
		error = -ENOMEM;
		goto err_array;
	}

	is_ram = region_intersects(res->start, resource_size(res),
		IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE);

	if (is_ram != REGION_DISJOINT) {
		WARN_ONCE(1, "%s attempted on %s region %pr\n", __func__,
				is_ram == REGION_MIXED ? "mixed" : "ram", res);
		error = -ENXIO;
		goto err_array;
	}

	error = xa_err(xa_store_range(&pgmap_array, PHYS_PFN(res->start),
				PHYS_PFN(res->end), pgmap, GFP_KERNEL));
	if (error)
		goto err_array;

	if (nid < 0)
		nid = numa_mem_id();

	error = track_pfn_remap(NULL, &pgprot, PHYS_PFN(res->start), 0,
			resource_size(res));
	if (error)
		goto err_pfn_remap;

	mem_hotplug_begin();

	/*
	 * For device private memory we call add_pages() as we only need to
	 * allocate and initialize struct page for the device memory. More-
	 * over the device memory is un-accessible thus we do not want to
	 * create a linear mapping for the memory like arch_add_memory()
	 * would do.
	 *
	 * For all other device memory types, which are accessible by
	 * the CPU, we do want the linear mapping and thus use
	 * arch_add_memory().
	 */
	if (pgmap->type == MEMORY_DEVICE_PRIVATE) {
		error = add_pages(nid, PHYS_PFN(res->start),
				PHYS_PFN(resource_size(res)), &restrictions);
	} else {
		error = kasan_add_zero_shadow(__va(res->start), resource_size(res));
		if (error) {
			mem_hotplug_done();
			goto err_kasan;
		}

		error = arch_add_memory(nid, res->start, resource_size(res),
					&restrictions);
	}

	if (!error) {
		struct zone *zone;

		zone = &NODE_DATA(nid)->node_zones[ZONE_DEVICE];
		move_pfn_range_to_zone(zone, PHYS_PFN(res->start),
				PHYS_PFN(resource_size(res)), restrictions.altmap);
	}

	mem_hotplug_done();
	if (error)
		goto err_add_memory;

	/*
	 * Initialization of the pages has been deferred until now in order
	 * to allow us to do the work while not holding the hotplug lock.
	 */
	memmap_init_zone_device(&NODE_DATA(nid)->node_zones[ZONE_DEVICE],
				PHYS_PFN(res->start),
				PHYS_PFN(resource_size(res)), pgmap);
	percpu_ref_get_many(pgmap->ref, pfn_end(pgmap) - pfn_first(pgmap));
	return __va(res->start);

 err_add_memory:
	kasan_remove_zero_shadow(__va(res->start), resource_size(res));
 err_kasan:
	untrack_pfn(NULL, PHYS_PFN(res->start), resource_size(res));
 err_pfn_remap:
	pgmap_array_delete(res);
 err_array:
	dev_pagemap_kill(pgmap);
	dev_pagemap_cleanup(pgmap);
	devmap_managed_enable_put();
	return ERR_PTR(error);
}
EXPORT_SYMBOL_GPL(memremap_pages);

/**
 * devm_memremap_pages - remap and provide memmap backing for the given resource
 * @dev: hosting device for @res
 * @pgmap: pointer to a struct dev_pagemap
 *
 * Notes:
 * 1/ At a minimum the res and type members of @pgmap must be initialized
 *    by the caller before passing it to this function
 *
 * 2/ The altmap field may optionally be initialized, in which case
 *    PGMAP_ALTMAP_VALID must be set in pgmap->flags.
 *
 * 3/ The ref field may optionally be provided, in which pgmap->ref must be
 *    'live' on entry and will be killed and reaped at
 *    devm_memremap_pages_release() time, or if this routine fails.
 *
 * 4/ res is expected to be a host memory range that could feasibly be
 *    treated as a "System RAM" range, i.e. not a device mmio range, but
 *    this is not enforced.
 */
void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)
{
	int error;
	void *ret;

	ret = memremap_pages(pgmap, dev_to_node(dev));
	if (IS_ERR(ret))
		return ret;

	error = devm_add_action_or_reset(dev, devm_memremap_pages_release,
			pgmap);
	if (error)
		return ERR_PTR(error);
	return ret;
}
EXPORT_SYMBOL_GPL(devm_memremap_pages);

void devm_memunmap_pages(struct device *dev, struct dev_pagemap *pgmap)
{
	devm_release_action(dev, devm_memremap_pages_release, pgmap);
}
EXPORT_SYMBOL_GPL(devm_memunmap_pages);

unsigned long vmem_altmap_offset(struct vmem_altmap *altmap)
{
	/* number of pfns from base where pfn_to_page() is valid */
	if (altmap)
		return altmap->reserve + altmap->free;
	return 0;
}

void vmem_altmap_free(struct vmem_altmap *altmap, unsigned long nr_pfns)
{
	altmap->alloc -= nr_pfns;
}

/**
 * get_dev_pagemap() - take a new live reference on the dev_pagemap for @pfn
 * @pfn: page frame number to lookup page_map
 * @pgmap: optional known pgmap that already has a reference
 *
 * If @pgmap is non-NULL and covers @pfn it will be returned as-is.  If @pgmap
 * is non-NULL but does not cover @pfn the reference to it will be released.
 */
struct dev_pagemap *get_dev_pagemap(unsigned long pfn,
		struct dev_pagemap *pgmap)
{
	resource_size_t phys = PFN_PHYS(pfn);

	/*
	 * In the cached case we're already holding a live reference.
	 */
	if (pgmap) {
		if (phys >= pgmap->res.start && phys <= pgmap->res.end)
			return pgmap;
		put_dev_pagemap(pgmap);
	}

	/* fall back to slow path lookup */
	rcu_read_lock();
	pgmap = xa_load(&pgmap_array, PHYS_PFN(phys));
	if (pgmap && !percpu_ref_tryget_live(pgmap->ref))
		pgmap = NULL;
	rcu_read_unlock();

	return pgmap;
}
EXPORT_SYMBOL_GPL(get_dev_pagemap);

#ifdef CONFIG_DEV_PAGEMAP_OPS
void __put_devmap_managed_page(struct page *page)
{
	int count = page_ref_dec_return(page);

	/*
	 * If refcount is 1 then page is freed and refcount is stable as nobody
	 * holds a reference on the page.
	 */
	if (count == 1) {
		/* Clear Active bit in case of parallel mark_page_accessed */
		__ClearPageActive(page);
		__ClearPageWaiters(page);

		mem_cgroup_uncharge(page);

		/*
		 * When a device_private page is freed, the page->mapping field
		 * may still contain a (stale) mapping value. For example, the
		 * lower bits of page->mapping may still identify the page as
		 * an anonymous page. Ultimately, this entire field is just
		 * stale and wrong, and it will cause errors if not cleared.
		 * One example is:
		 *
		 *  migrate_vma_pages()
		 *    migrate_vma_insert_page()
		 *      page_add_new_anon_rmap()
		 *        __page_set_anon_rmap()
		 *          ...checks page->mapping, via PageAnon(page) call,
		 *            and incorrectly concludes that the page is an
		 *            anonymous page. Therefore, it incorrectly,
		 *            silently fails to set up the new anon rmap.
		 *
		 * For other types of ZONE_DEVICE pages, migration is either
		 * handled differently or not done at all, so there is no need
		 * to clear page->mapping.
		 */
		if (is_device_private_page(page))
			page->mapping = NULL;

		page->pgmap->ops->page_free(page);
	} else if (!count)
		__put_page(page);
}
EXPORT_SYMBOL(__put_devmap_managed_page);
#endif /* CONFIG_DEV_PAGEMAP_OPS */
反馈
建议
客服 返回
顶部