1. 20 10月, 2018 13 次提交
    • S
      bpf: add cg_skb_is_valid_access for BPF_PROG_TYPE_CGROUP_SKB · b39b5f41
      Song Liu 提交于
      BPF programs of BPF_PROG_TYPE_CGROUP_SKB need to access headers in the
      skb. This patch enables direct access of skb for these programs.
      
      Two helper functions bpf_compute_and_save_data_end() and
      bpf_restore_data_end() are introduced. There are used in
      __cgroup_bpf_run_filter_skb(), to compute proper data_end for the
      BPF program, and restore original data afterwards.
      Signed-off-by: NSong Liu <songliubraving@fb.com>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      b39b5f41
    • A
      Merge branch 'improve_perf_barriers' · 2929ad29
      Alexei Starovoitov 提交于
      Daniel Borkmann says:
      
      ====================
      This set first adds smp_* barrier variants to tools infrastructure
      and updates perf and libbpf to make use of them. For details, please
      see individual patches, thanks!
      
      Arnaldo, if there are no objections, could this be routed via bpf-next
      with Acked-by's due to later dependencies in libbpf? Alternatively,
      I could also get the 2nd patch out during merge window, but perhaps
      it's okay to do in one go as there shouldn't be much conflict in perf
      itself.
      
      Thanks!
      
      v1 -> v2:
        - add common helper and switch to acquire/release variants
          when possible, thanks Peter!
      ====================
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      2929ad29
    • D
      bpf, libbpf: use correct barriers in perf ring buffer walk · a64af0ef
      Daniel Borkmann 提交于
      Given libbpf is a generic library and not restricted to x86-64 only,
      the compiler barrier in bpf_perf_event_read_simple() after fetching
      the head needs to be replaced with smp_rmb() at minimum. Also, writing
      out the tail we should use WRITE_ONCE() to avoid store tearing.
      
      Now that we have the logic in place in ring_buffer_read_head() and
      ring_buffer_write_tail() helper also used by perf tool which would
      select the correct and best variant for a given architecture (e.g.
      x86-64 can avoid CPU barriers entirely), make use of these in order
      to fix bpf_perf_event_read_simple().
      
      Fixes: d0cabbb0 ("tools: bpf: move the event reading loop to libbpf")
      Fixes: 39111695 ("samples: bpf: add bpf_perf_event_output example")
      Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      a64af0ef
    • D
      tools, perf: add and use optimized ring_buffer_{read_head, write_tail} helpers · 09d62154
      Daniel Borkmann 提交于
      Currently, on x86-64, perf uses LFENCE and MFENCE (rmb() and mb(),
      respectively) when processing events from the perf ring buffer which
      is unnecessarily expensive as we can do more lightweight in particular
      given this is critical fast-path in perf.
      
      According to Peter rmb()/mb() were added back then via a94d342b
      ("tools/perf: Add required memory barriers") at a time where kernel
      still supported chips that needed it, but nowadays support for these
      has been ditched completely, therefore we can fix them up as well.
      
      While for x86-64, replacing rmb() and mb() with smp_*() variants would
      result in just a compiler barrier for the former and LOCK + ADD for
      the latter (__sync_synchronize() uses slower MFENCE by the way), Peter
      suggested we can use smp_{load_acquire,store_release}() instead for
      architectures where its implementation doesn't resolve in slower smp_mb().
      Thus, e.g. in x86-64 we would be able to avoid CPU barrier entirely due
      to TSO. For architectures where the latter needs to use smp_mb() e.g.
      on arm, we stick to cheaper smp_rmb() variant for fetching the head.
      
      This work adds helpers ring_buffer_read_head() and ring_buffer_write_tail()
      for tools infrastructure that either switches to smp_load_acquire() for
      architectures where it is cheaper or uses READ_ONCE() + smp_rmb() barrier
      for those where it's not in order to fetch the data_head from the perf
      control page, and it uses smp_store_release() to write the data_tail.
      Latter is smp_mb() + WRITE_ONCE() combination or a cheaper variant if
      architecture allows for it. Those that rely on smp_rmb() and smp_mb() can
      further improve performance in a follow up step by implementing the two
      under tools/arch/*/include/asm/barrier.h such that they don't have to
      fallback to rmb() and mb() in tools/include/asm/barrier.h.
      
      Switch perf to use ring_buffer_read_head() and ring_buffer_write_tail()
      so it can make use of the optimizations. Later, we convert libbpf as
      well to use the same helpers.
      
      Side note [0]: the topic has been raised of whether one could simply use
      the C11 gcc builtins [1] for the smp_load_acquire() and smp_store_release()
      instead:
      
        __atomic_load_n(ptr, __ATOMIC_ACQUIRE);
        __atomic_store_n(ptr, val, __ATOMIC_RELEASE);
      
      Kernel and (presumably) tooling shipped along with the kernel has a
      minimum requirement of being able to build with gcc-4.6 and the latter
      does not have C11 builtins. While generally the C11 memory models don't
      align with the kernel's, the C11 load-acquire and store-release alone
      /could/ suffice, however. Issue is that this is implementation dependent
      on how the load-acquire and store-release is done by the compiler and
      the mapping of supported compilers must align to be compatible with the
      kernel's implementation, and thus needs to be verified/tracked on a
      case by case basis whether they match (unless an architecture uses them
      also from kernel side). The implementations for smp_load_acquire() and
      smp_store_release() in this patch have been adapted from the kernel side
      ones to have a concrete and compatible mapping in place.
      
        [0] http://patchwork.ozlabs.org/patch/985422/
        [1] https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.htmlSigned-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: NPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      09d62154
    • A
      selftests/bpf: add missing executables to .gitignore · 78de3546
      Anders Roxell 提交于
      Fixes: 371e4fcc ("selftests/bpf: cgroup local storage-based network counters")
      Fixes: 370920c4 ("selftests/bpf: Test libbpf_{prog,attach}_type_by_name")
      Signed-off-by: NAnders Roxell <anders.roxell@linaro.org>
      Acked-by: NYonghong Song <yhs@fb.com>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      78de3546
    • A
      Merge branch 'queue_stack_maps' · 43ed375f
      Alexei Starovoitov 提交于
      Mauricio Vasquez says:
      
      ====================
      In some applications this is needed have a pool of free elements, for
      example the list of free L4 ports in a SNAT.  None of the current maps allow
      to do it as it is not possible to get any element without having they key
      it is associated to, even if it were possible, the lack of locking mecanishms in
      eBPF would do it almost impossible to be implemented without data races.
      
      This patchset implements two new kind of eBPF maps: queue and stack.
      Those maps provide to eBPF programs the peek, push and pop operations, and for
      userspace applications a new bpf_map_lookup_and_delete_elem() is added.
      Signed-off-by: NMauricio Vasquez B <mauricio.vasquez@polito.it>
      
      v2 -> v3:
       - Remove "almost dead code" in syscall.c
       - Remove unnecessary copy_from_user in bpf_map_lookup_and_delete_elem
       - Rebase
      
      v1 -> v2:
       - Put ARG_PTR_TO_UNINIT_MAP_VALUE logic into a separated patch
       - Fix missing __this_cpu_dec & preempt_enable calls in kernel/bpf/syscall.c
      
      RFC v4 -> v1:
       - Remove roundup to power of 2 in memory allocation
       - Remove count and use a free slot to check if queue/stack is empty
       - Use if + assigment for wrapping indexes
       - Fix some minor style issues
       - Squash two patches together
      
      RFC v3 -> RFC v4:
       - Revert renaming of kernel/bpf/stackmap.c
       - Remove restriction on value size
       - Remove len arguments from peek/pop helpers
       - Add new ARG_PTR_TO_UNINIT_MAP_VALUE
      
      RFC v2 -> RFC v3:
       - Return elements by value instead that by reference
       - Implement queue/stack base on array and head + tail indexes
       - Rename stack trace related files to avoid confusion and conflicts
      
      RFC v1 -> RFC v2:
       - Create two separate maps instead of single one + flags
       - Implement bpf_map_lookup_and_delete syscall
       - Support peek operation
       - Define replacement policy through flags in the update() method
       - Add eBPF side tests
      ====================
      Acked-by: NDaniel Borkmann <daniel@iogearbox.net>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      43ed375f
    • M
      selftests/bpf: add test cases for queue and stack maps · 43b987d2
      Mauricio Vasquez B 提交于
      test_maps:
      Tests that queue/stack maps are behaving correctly even in corner cases
      
      test_progs:
      Tests new ebpf helpers
      Signed-off-by: NMauricio Vasquez B <mauricio.vasquez@polito.it>
      Acked-by: NSong Liu <songliubraving@fb.com>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      43b987d2
    • M
      Sync uapi/bpf.h to tools/include · da4e1b15
      Mauricio Vasquez B 提交于
      Sync both files.
      Signed-off-by: NMauricio Vasquez B <mauricio.vasquez@polito.it>
      Acked-by: NSong Liu <songliubraving@fb.com>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      da4e1b15
    • M
      bpf: add MAP_LOOKUP_AND_DELETE_ELEM syscall · bd513cd0
      Mauricio Vasquez B 提交于
      The previous patch implemented a bpf queue/stack maps that
      provided the peek/pop/push functions.  There is not a direct
      relationship between those functions and the current maps
      syscalls, hence a new MAP_LOOKUP_AND_DELETE_ELEM syscall is added,
      this is mapped to the pop operation in the queue/stack maps
      and it is still to implement in other kind of maps.
      Signed-off-by: NMauricio Vasquez B <mauricio.vasquez@polito.it>
      Acked-by: NSong Liu <songliubraving@fb.com>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      bd513cd0
    • M
      bpf: add queue and stack maps · f1a2e44a
      Mauricio Vasquez B 提交于
      Queue/stack maps implement a FIFO/LIFO data storage for ebpf programs.
      These maps support peek, pop and push operations that are exposed to eBPF
      programs through the new bpf_map[peek/pop/push] helpers.  Those operations
      are exposed to userspace applications through the already existing
      syscalls in the following way:
      
      BPF_MAP_LOOKUP_ELEM            -> peek
      BPF_MAP_LOOKUP_AND_DELETE_ELEM -> pop
      BPF_MAP_UPDATE_ELEM            -> push
      
      Queue/stack maps are implemented using a buffer, tail and head indexes,
      hence BPF_F_NO_PREALLOC is not supported.
      
      As opposite to other maps, queue and stack do not use RCU for protecting
      maps values, the bpf_map[peek/pop] have a ARG_PTR_TO_UNINIT_MAP_VALUE
      argument that is a pointer to a memory zone where to save the value of a
      map.  Basically the same as ARG_PTR_TO_UNINIT_MEM, but the size has not
      be passed as an extra argument.
      
      Our main motivation for implementing queue/stack maps was to keep track
      of a pool of elements, like network ports in a SNAT, however we forsee
      other use cases, like for exampling saving last N kernel events in a map
      and then analysing from userspace.
      Signed-off-by: NMauricio Vasquez B <mauricio.vasquez@polito.it>
      Acked-by: NSong Liu <songliubraving@fb.com>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      f1a2e44a
    • M
      bpf/verifier: add ARG_PTR_TO_UNINIT_MAP_VALUE · 2ea864c5
      Mauricio Vasquez B 提交于
      ARG_PTR_TO_UNINIT_MAP_VALUE argument is a pointer to a memory zone
      used to save the value of a map.  Basically the same as
      ARG_PTR_TO_UNINIT_MEM, but the size has not be passed as an extra
      argument.
      
      This will be used in the following patch that implements some new
      helpers that receive a pointer to be filled with a map value.
      Signed-off-by: NMauricio Vasquez B <mauricio.vasquez@polito.it>
      Acked-by: NSong Liu <songliubraving@fb.com>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      2ea864c5
    • M
      bpf/syscall: allow key to be null in map functions · c9d29f46
      Mauricio Vasquez B 提交于
      This commit adds the required logic to allow key being NULL
      in case the key_size of the map is 0.
      
      A new __bpf_copy_key function helper only copies the key from
      userpsace when key_size != 0, otherwise it enforces that key must be
      null.
      Signed-off-by: NMauricio Vasquez B <mauricio.vasquez@polito.it>
      Acked-by: NSong Liu <songliubraving@fb.com>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      c9d29f46
    • M
      bpf: rename stack trace map operations · 14499160
      Mauricio Vasquez B 提交于
      In the following patches queue and stack maps (FIFO and LIFO
      datastructures) will be implemented.  In order to avoid confusion and
      a possible name clash rename stack_map_ops to stack_trace_map_ops
      Signed-off-by: NMauricio Vasquez B <mauricio.vasquez@polito.it>
      Acked-by: NSong Liu <songliubraving@fb.com>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      14499160
  2. 19 10月, 2018 2 次提交
  3. 18 10月, 2018 1 次提交
  4. 17 10月, 2018 9 次提交
  5. 16 10月, 2018 15 次提交
    • D
      bpf, tls: add tls header to tools infrastructure · 421f4292
      Daniel Borkmann 提交于
      Andrey reported a build error for the BPF kselftest suite when compiled on
      a machine which does not have tls related header bits installed natively:
      
        test_sockmap.c:120:23: fatal error: linux/tls.h: No such file or directory
         #include <linux/tls.h>
                               ^
        compilation terminated.
      
      Fix it by adding the header to the tools include infrastructure and add
      definitions such as SOL_TLS that could potentially be missing.
      
      Fixes: e9dd9047 ("bpf: add tls support for testing in test_sockmap")
      Reported-by: NAndrey Ignatov <rdna@fb.com>
      Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      421f4292
    • D
      Merge branch 'net-Kernel-side-filtering-for-route-dumps' · 2c59f06c
      David S. Miller 提交于
      David Ahern says:
      
      ====================
      net: Kernel side filtering for route dumps
      
      Implement kernel side filtering of route dumps by protocol (e.g., which
      routing daemon installed the route), route type (e.g., unicast), table
      id and nexthop device.
      
      iproute2 has been doing this filtering in userspace for years; pushing
      the filters to the kernel side reduces the amount of data the kernel
      sends and reduces wasted cycles on both sides processing unwanted data.
      These initial options provide a huge improvement for efficiently
      examining routes on large scale systems.
      
      v2
      - better handling of requests for a specific table. Rather than walking
        the hash of all tables, lookup the specific table and dump it
      - refactor mr_rtm_dumproute moving the loop over the table into a
        helper that can be invoked directly
      - add hook to return NLM_F_DUMP_FILTERED in DONE message to ensure
        it is returned even when the dump returns nothing
      ====================
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      2c59f06c
    • D
      net/ipv4: Bail early if user only wants prefix entries · e4e92fb1
      David Ahern 提交于
      Unlike IPv6, IPv4 does not have routes marked with RTF_PREFIX_RT. If the
      flag is set in the dump request, just return.
      
      In the process of this change, move the CLONE check to use the new
      filter flags.
      Signed-off-by: NDavid Ahern <dsahern@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      e4e92fb1
    • D
      net/ipv6: Bail early if user only wants cloned entries · 08e814c9
      David Ahern 提交于
      Similar to IPv4, IPv6 fib no longer contains cloned routes. If a user
      requests a route dump for only cloned entries, no sense walking the FIB
      and returning everything.
      Signed-off-by: NDavid Ahern <dsahern@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      08e814c9
    • D
      net/mpls: Handle kernel side filtering of route dumps · 196cfebf
      David Ahern 提交于
      Update the dump request parsing in MPLS for the non-INET case to
      enable kernel side filtering. If INET is disabled the only filters
      that make sense for MPLS are protocol and nexthop device.
      Signed-off-by: NDavid Ahern <dsahern@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      196cfebf
    • D
      net: Enable kernel side filtering of route dumps · effe6792
      David Ahern 提交于
      Update parsing of route dump request to enable kernel side filtering.
      Allow filtering results by protocol (e.g., which routing daemon installed
      the route), route type (e.g., unicast), table id and nexthop device. These
      amount to the low hanging fruit, yet a huge improvement, for dumping
      routes.
      
      ip_valid_fib_dump_req is called with RTNL held, so __dev_get_by_index can
      be used to look up the device index without taking a reference. From
      there filter->dev is only used during dump loops with the lock still held.
      
      Set NLM_F_DUMP_FILTERED in the answer_flags so the user knows the results
      have been filtered should no entries be returned.
      Signed-off-by: NDavid Ahern <dsahern@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      effe6792
    • D
      net: Plumb support for filtering ipv4 and ipv6 multicast route dumps · cb167893
      David Ahern 提交于
      Implement kernel side filtering of routes by egress device index and
      table id. If the table id is given in the filter, lookup table and
      call mr_table_dump directly for it.
      Signed-off-by: NDavid Ahern <dsahern@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      cb167893
    • D
      ipmr: Refactor mr_rtm_dumproute · e1cedae1
      David Ahern 提交于
      Move per-table loops from mr_rtm_dumproute to mr_table_dump and export
      mr_table_dump for dumps by specific table id.
      Signed-off-by: NDavid Ahern <dsahern@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      e1cedae1
    • D
      net/mpls: Plumb support for filtering route dumps · bae9a78b
      David Ahern 提交于
      Implement kernel side filtering of routes by egress device index and
      protocol. MPLS uses only a single table and route type.
      Signed-off-by: NDavid Ahern <dsahern@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      bae9a78b
    • D
      net/ipv6: Plumb support for filtering route dumps · 13e38901
      David Ahern 提交于
      Implement kernel side filtering of routes by table id, egress device
      index, protocol, and route type. If the table id is given in the filter,
      lookup the table and call fib6_dump_table directly for it.
      
      Move the existing route flags check for prefix only routes to the new
      filter.
      Signed-off-by: NDavid Ahern <dsahern@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      13e38901
    • D
      net/ipv4: Plumb support for filtering route dumps · 18a8021a
      David Ahern 提交于
      Implement kernel side filtering of routes by table id, egress device index,
      protocol and route type. If the table id is given in the filter, lookup the
      table and call fib_table_dump directly for it.
      Signed-off-by: NDavid Ahern <dsahern@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      18a8021a
    • D
      net: Add struct for fib dump filter · 4724676d
      David Ahern 提交于
      Add struct fib_dump_filter for options on limiting which routes are
      returned in a dump request. The current list is table id, protocol,
      route type, rtm_flags and nexthop device index. struct net is needed
      to lookup the net_device from the index.
      
      Declare the filter for each route dump handler and plumb the new
      arguments from dump handlers to ip_valid_fib_dump_req.
      Signed-off-by: NDavid Ahern <dsahern@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      4724676d
    • D
      netlink: Add answer_flags to netlink_callback · 22e6c58b
      David Ahern 提交于
      With dump filtering we need a way to ensure the NLM_F_DUMP_FILTERED
      flag is set on a message back to the user if the data returned is
      influenced by some input attributes. Normally this can be done as
      messages are added to the skb, but if the filter results in no data
      being returned, the user could be confused as to why.
      
      This patch adds answer_flags to the netlink_callback allowing dump
      handlers to set the NLM_F_DUMP_FILTERED at a minimum in the
      NLMSG_DONE message ensuring the flag gets back to the user.
      
      The netlink_callback space is initialized to 0 via a memset in
      __netlink_dump_start, so init of the new answer_flags is covered.
      Signed-off-by: NDavid Ahern <dsahern@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      22e6c58b
    • D
      Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next · e8567951
      David S. Miller 提交于
      Daniel Borkmann says:
      
      ====================
      pull-request: bpf-next 2018-10-16
      
      The following pull-request contains BPF updates for your *net-next* tree.
      
      The main changes are:
      
      1) Convert BPF sockmap and kTLS to both use a new sk_msg API and enable
         sk_msg BPF integration for the latter, from Daniel and John.
      
      2) Enable BPF syscall side to indicate for maps that they do not support
         a map lookup operation as opposed to just missing key, from Prashant.
      
      3) Add bpftool map create command which after map creation pins the
         map into bpf fs for further processing, from Jakub.
      
      4) Add bpftool support for attaching programs to maps allowing sock_map
         and sock_hash to be used from bpftool, from John.
      
      5) Improve syscall BPF map update/delete path for map-in-map types to
         wait a RCU grace period for pending references to complete, from Daniel.
      
      6) Couple of follow-up fixes for the BPF socket lookup to get it
         enabled also when IPv6 is compiled as a module, from Joe.
      
      7) Fix a generic-XDP bug to handle the case when the Ethernet header
         was mangled and thus update skb's protocol and data, from Jesper.
      
      8) Add a missing BTF header length check between header copies from
         user space, from Wenwen.
      
      9) Minor fixups in libbpf to use __u32 instead u32 types and include
         proper perf_event.h uapi header instead of perf internal one, from Yonghong.
      
      10) Allow to pass user-defined flags through EXTRA_CFLAGS and EXTRA_LDFLAGS
          to bpftool's build, from Jiri.
      
      11) BPF kselftest tweaks to add LWTUNNEL to config fragment and to install
          with_addr.sh script from flow dissector selftest, from Anders.
      ====================
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      e8567951
    • H
      net: phy: merge phy_start_aneg and phy_start_aneg_priv · c45d7150
      Heiner Kallweit 提交于
      After commit 9f2959b6 ("net: phy: improve handling delayed work")
      the sync parameter isn't needed any longer in phy_start_aneg_priv().
      This allows to merge phy_start_aneg() and phy_start_aneg_priv().
      Signed-off-by: NHeiner Kallweit <hkallweit1@gmail.com>
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      c45d7150