Vlastimil Babka 56afe477df mm: munlock: bypass per-cpu pvec for putback_lru_page ...

After introducing batching by pagevecs into munlock_vma_range(), we can
further improve performance by bypassing the copying into per-cpu pagevec
and the get_page/put_page pair associated with that.  Instead we perform
LRU putback directly from our pagevec.  However, this is possible only for
single-mapped pages that are evictable after munlock.  Unevictable pages
require rechecking after putting on the unevictable list, so for those we
fallback to putback_lru_page(), hich handles that.

After this patch, a 13% speedup was measured for munlocking a 56GB large
memory area with THP disabled.

[akpm@linux-foundation.org:clarify comment]
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Reviewed-by: Jörn Engel <joern@logfs.org>
Acked-by: Mel Gorman <mgorman@suse.de>
Cc: Michel Lespinasse <walken@google.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

2013-09-11 15:58:00 -07:00

..

backing-dev.c

backing-dev: convert class code to use dev_groups

2013-08-19 21:22:34 -07:00

balloon_compaction.c

…

bootmem.c

mm: kill free_all_bootmem_node()

2013-07-03 16:07:39 -07:00

bounce.c

Merge branch 'for-3.10/core' of git://git.kernel.dk/linux-block

2013-05-08 10:13:35 -07:00

cleancache.c

mm: cleancache: clean up cleancache_enabled

2013-04-30 17:04:01 -07:00

compaction.c

mm: compaction: do not compact pgdat for order-0

2013-09-11 15:57:55 -07:00

debug-pagealloc.c

…

dmapool.c

…

fadvise.c

teach SYSCALL_DEFINE<n> how to deal with long long/unsigned long long

2013-03-03 22:46:22 -05:00

failslab.c

…

filemap_xip.c

lift sb_start_write() out of ->write()

2013-04-09 14:12:56 -04:00

filemap.c

direct-io: Handle O_(D)SYNC AIO

2013-09-04 09:23:46 -04:00

fremap.c

mm: save soft-dirty bits on file pages

2013-08-13 17:57:48 -07:00

frontswap.c

frontswap: fix incorrect zeroing and allocation size for frontswap_map

2013-06-12 16:29:46 -07:00

highmem.c

Merge tag 'virtio-next-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux

2012-12-20 08:37:05 -08:00

huge_memory.c

mm/huge_memory.c: fix potential NULL pointer dereference

2013-09-11 15:57:19 -07:00

hugetlb_cgroup.c

cgroup: pass around cgroup_subsys_state instead of cgroup in file methods

2013-08-08 20:11:24 -04:00

hugetlb.c

mm: prepare to remove /proc/sys/vm/hugepages_treat_as_movable

2013-09-11 15:57:49 -07:00

hwpoison-inject.c

…

init-mm.c

…

internal.h

mm: remove unused __put_page()

2013-07-09 10:33:22 -07:00

interval_tree.c

…

Kconfig

Merge remote-tracking branch 'origin/next' into kvm-ppc-next

2013-08-29 00:41:59 +02:00

Kconfig.debug

…

kmemcheck.c

…

kmemleak-test.c

…

kmemleak.c

mm: replace strict_strtoul() with kstrtoul()

2013-09-11 15:57:11 -07:00

ksm.c

mm: replace strict_strtoul() with kstrtoul()

2013-09-11 15:57:11 -07:00

maccess.c

…

madvise.c

mm/madvise.c: fix coding-style errors

2013-09-11 15:57:00 -07:00

Makefile

zswap: add to mm/

2013-07-10 18:11:34 -07:00

memblock.c

memblock, numa: binary search node id

2013-09-11 15:57:51 -07:00

memcontrol.c

kmemcg: don't allocate extra memory for root memcg_cache_params

2013-09-11 15:57:53 -07:00

memory_hotplug.c

mm: memory-hotplug: enable memory hotplug to handle hugepage

2013-09-11 15:57:48 -07:00

memory-failure.c

mm: soft-offline: use migrate_pages() instead of migrate_huge_page()

2013-09-11 15:57:47 -07:00

memory.c

mm: migrate: add hugepage migration code to move_pages()

2013-09-11 15:57:48 -07:00

mempolicy.c

mbind: add BUG_ON(!vma) in new_vma_page()

2013-09-11 15:57:50 -07:00

mempool.c

…

migrate.c

mm: migrate: check movability of hugepage in unmap_and_move_huge_page()

2013-09-11 15:57:49 -07:00

mincore.c

swap: make each swap partition have one address_space

2013-02-23 17:50:17 -08:00

mlock.c

mm: munlock: bypass per-cpu pvec for putback_lru_page

2013-09-11 15:58:00 -07:00

mm_init.c

mm: tune vm_committed_as percpu_counter batching size

2013-07-03 16:07:32 -07:00

mmap.c

mm: track vma changes with VM_SOFTDIRTY bit

2013-09-11 15:57:56 -07:00

mmu_context.c

mm: remove old aio use_mm() comment

2013-05-07 18:38:27 -07:00

mmu_notifier.c

treewide: relase -> release

2013-06-28 14:34:33 +02:00

mmzone.c

mm: rename page struct field helpers

2013-02-23 17:50:18 -08:00

mprotect.c

mm/mprotect.c: coding-style cleanups

2012-12-18 15:02:15 -08:00

mremap.c

mm: move_ptes -- Set soft dirty bit depending on pte type

2013-08-27 09:36:17 -07:00

msync.c

…

nobootmem.c

mm: concentrate modification of totalram_pages into the mm core

2013-07-03 16:07:33 -07:00

nommu.c

mm: remove free_area_cache

2013-07-10 18:11:34 -07:00

oom_kill.c

mm/oom_kill: remove weird use of ERR_PTR()/PTR_ERR().

2013-07-15 11:25:05 +09:30

page_alloc.c

mm: page_alloc: fix comment get_page_from_freelist

2013-09-11 15:57:56 -07:00

page_cgroup.c

memcontrol: use N_MEMORY instead N_HIGH_MEMORY

2012-12-12 17:38:32 -08:00

page_io.c

mm: remove compressed copy from zram in-memory

2013-07-03 16:07:26 -07:00

page_isolation.c

mm: memory-hotplug: enable memory hotplug to handle hugepage

2013-09-11 15:57:48 -07:00

page-writeback.c

mm: revert "page-writeback.c: subtract min_free_kbytes from dirtyable memory"

2013-09-11 15:57:23 -07:00

pagewalk.c

mm/pagewalk.c: walk_page_range should avoid VM_PFNMAP areas

2013-05-24 16:22:53 -07:00

percpu-km.c

…

percpu-vm.c

…

percpu.c

…

pgtable-generic.c

mm: move pgtable related functions to right place

2013-09-11 15:57:30 -07:00

process_vm_access.c

Fix: compat_rw_copy_check_uvector() misuse in aio, readv, writev, and security keys

2013-03-12 11:05:45 -07:00

quicklist.c

…

readahead.c

readahead: make context readahead more conservative

2013-09-11 15:57:39 -07:00

rmap.c

s390/mm: implement software referenced bits

2013-08-29 13:20:11 +02:00

shmem.c

shm_mnt is as longterm as it gets, TYVM...

2013-09-03 22:50:27 -04:00

slab_common.c

Merge branch 'slab/for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/linux

2013-07-14 15:14:29 -07:00

slab.c

kernel: delete __cpuinit usage from all core kernel files

2013-07-14 19:36:59 -04:00

slab.h

memcg: check that kmem_cache has memcg_params before accessing it

2013-08-28 19:26:38 -07:00

slob.c

Merge branch 'slab/for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/linux

2013-07-14 15:14:29 -07:00

slub.c

mm: replace strict_strtoul() with kstrtoul()

2013-09-11 15:57:11 -07:00

sparse-vmemmap.c

sparse-vmemmap: specify vmemmap population range in bytes

2013-04-29 15:54:35 -07:00

sparse.c

mm/sparse.c: put clear_hwpoisoned_pages within CONFIG_MEMORY_HOTREMOVE

2013-07-09 10:33:22 -07:00

swap_state.c

swap: avoid read_swap_cache_async() race to deadlock while waiting on discard I/O completion

2013-06-12 16:29:45 -07:00

swap.c

mm: fix aio performance regression for database caused by THP

2013-09-11 15:57:55 -07:00

swapfile.c

swap: make cluster allocation per-cpu

2013-09-11 15:57:17 -07:00

truncate.c

mm: teach truncate_inode_pages_range() to handle non page aligned ranges

2013-05-27 23:32:35 -04:00

util.c

swap: clean-up #ifdef in page_mapping()

2013-09-11 15:57:31 -07:00

vmalloc.c

mm, vmalloc: use well-defined find_last_bit() func

2013-09-11 15:57:34 -07:00

vmpressure.c

Merge branch 'for-3.12' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup

2013-09-03 18:25:03 -07:00

vmscan.c

mm: putback_lru_page: remove unnecessary call to page_lru_base_type()

2013-09-11 15:57:57 -07:00

vmstat.c

vmstat: use this_cpu() to avoid irqon/off sequence in refresh_cpu_vm_stats

2013-09-11 15:57:31 -07:00

zbud.c

mm/zbud: fix some trivial typos in comments

2013-09-11 15:57:35 -07:00

zswap.c

mm/zswap.c: get swapper address_space by using macro

2013-09-11 15:57:08 -07:00