1. 21 12月, 2017 1 次提交
    • D
      bpf: fix kallsyms handling for subprogs · 4f74d809
      Daniel Borkmann 提交于
      Right now kallsyms handling is not working with JITed subprogs.
      The reason is that when in 1c2a088a ("bpf: x64: add JIT support
      for multi-function programs") in jit_subprogs() they are passed
      to bpf_prog_kallsyms_add(), then their prog type is 0, which BPF
      core will think it's a cBPF program as only cBPF programs have a
      0 type. Thus, they need to inherit the type from the main prog.
      
      Once that is fixed, they are indeed added to the BPF kallsyms
      infra, but their tag is 0. Therefore, since intention is to add
      them as bpf_prog_F_<tag>, we need to pass them to bpf_prog_calc_tag()
      first. And once this is resolved, there is a use-after-free on
      prog cleanup: we remove the kallsyms entry from the main prog,
      later walk all subprogs and call bpf_jit_free() on them. However,
      the kallsyms linkage was never released on them. Thus, do that
      for all subprogs right in __bpf_prog_put() when refcount hits 0.
      
      Fixes: 1c2a088a ("bpf: x64: add JIT support for multi-function programs")
      Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: NAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      4f74d809
  2. 20 12月, 2017 1 次提交
  3. 19 12月, 2017 30 次提交
  4. 18 12月, 2017 8 次提交
    • D
      Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next · 59436c9e
      David S. Miller 提交于
      Daniel Borkmann says:
      
      ====================
      pull-request: bpf-next 2017-12-18
      
      The following pull-request contains BPF updates for your *net-next* tree.
      
      The main changes are:
      
      1) Allow arbitrary function calls from one BPF function to another BPF function.
         As of today when writing BPF programs, __always_inline had to be used in
         the BPF C programs for all functions, unnecessarily causing LLVM to inflate
         code size. Handle this more naturally with support for BPF to BPF calls
         such that this __always_inline restriction can be overcome. As a result,
         it allows for better optimized code and finally enables to introduce core
         BPF libraries in the future that can be reused out of different projects.
         x86 and arm64 JIT support was added as well, from Alexei.
      
      2) Add infrastructure for tagging functions as error injectable and allow for
         BPF to return arbitrary error values when BPF is attached via kprobes on
         those. This way of injecting errors generically eases testing and debugging
         without having to recompile or restart the kernel. Tags for opting-in for
         this facility are added with BPF_ALLOW_ERROR_INJECTION(), from Josef.
      
      3) For BPF offload via nfp JIT, add support for bpf_xdp_adjust_head() helper
         call for XDP programs. First part of this work adds handling of BPF
         capabilities included in the firmware, and the later patches add support
         to the nfp verifier part and JIT as well as some small optimizations,
         from Jakub.
      
      4) The bpftool now also gets support for basic cgroup BPF operations such
         as attaching, detaching and listing current BPF programs. As a requirement
         for the attach part, bpftool can now also load object files through
         'bpftool prog load'. This reuses libbpf which we have in the kernel tree
         as well. bpftool-cgroup man page is added along with it, from Roman.
      
      5) Back then commit e87c6bc3 ("bpf: permit multiple bpf attachments for
         a single perf event") added support for attaching multiple BPF programs
         to a single perf event. Given they are configured through perf's ioctl()
         interface, the interface has been extended with a PERF_EVENT_IOC_QUERY_BPF
         command in this work in order to return an array of one or multiple BPF
         prog ids that are currently attached, from Yonghong.
      
      6) Various minor fixes and cleanups to the bpftool's Makefile as well
         as a new 'uninstall' and 'doc-uninstall' target for removing bpftool
         itself or prior installed documentation related to it, from Quentin.
      
      7) Add CONFIG_CGROUP_BPF=y to the BPF kernel selftest config file which is
         required for the test_dev_cgroup test case to run, from Naresh.
      
      8) Fix reporting of XDP prog_flags for nfp driver, from Jakub.
      
      9) Fix libbpf's exit code from the Makefile when libelf was not found in
         the system, also from Jakub.
      ====================
      Signed-off-by: NDavid S. Miller <davem@davemloft.net>
      59436c9e
    • J
      trace: reenable preemption if we modify the ip · 46df3d20
      Josef Bacik 提交于
      Things got moved around between the original bpf_override_return patches
      and the final version, and now the ftrace kprobe dispatcher assumes if
      you modified the ip that you also enabled preemption.  Make a comment of
      this and enable preemption, this fixes the lockdep splat that happened
      when using this feature.
      
      Fixes: 9802d865 ("bpf: add a bpf_override_function helper")
      Signed-off-by: NJosef Bacik <jbacik@fb.com>
      Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      46df3d20
    • J
      nfp: set flags in the correct member of netdev_bpf · 4a29c0db
      Jakub Kicinski 提交于
      netdev_bpf.flags is the input member for installing the program.
      netdev_bpf.prog_flags is the output member for querying.  Set
      the correct one on query.
      
      Fixes: 92f0292b ("net: xdp: report flags program was installed with on query")
      Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
      Reviewed-by: NQuentin Monnet <quentin.monnet@netronome.com>
      Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      4a29c0db
    • J
      libbpf: fix Makefile exit code if libelf not found · 21567ede
      Jakub Kicinski 提交于
      /bin/sh's exit does not recognize -1 as a number, leading to
      the following error message:
      
      /bin/sh: 1: exit: Illegal number: -1
      
      Use 1 as the exit code.
      Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
      Reviewed-by: NQuentin Monnet <quentin.monnet@netronome.com>
      Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      21567ede
    • D
      Merge branch 'bpf-to-bpf-function-calls' · ef9fde06
      Daniel Borkmann 提交于
      Alexei Starovoitov says:
      
      ====================
      First of all huge thank you to Daniel, John, Jakub, Edward and others who
      reviewed multiple iterations of this patch set over the last many months
      and to Dave and others who gave critical feedback during netconf/netdev.
      
      The patch is solid enough and we thought through numerous corner cases,
      but it's not the end. More followups with code reorg and features to follow.
      
      TLDR: Allow arbitrary function calls from bpf function to another bpf function.
      
      Since the beginning of bpf all bpf programs were represented as a single function
      and program authors were forced to use always_inline for all functions
      in their C code. That was causing llvm to unnecessary inflate the code size
      and forcing developers to move code to header files with little code reuse.
      
      With a bit of additional complexity teach verifier to recognize
      arbitrary function calls from one bpf function to another as long as
      all of functions are presented to the verifier as a single bpf program.
      Extended program layout:
      ..
      r1 = ..    // arg1
      r2 = ..    // arg2
      call pc+1  // function call pc-relative
      exit
      .. = r1    // access arg1
      .. = r2    // access arg2
      ..
      call pc+20 // second level of function call
      ...
      
      It allows for better optimized code and finally allows to introduce
      the core bpf libraries that can be reused in different projects,
      since programs are no longer limited by single elf file.
      With function calls bpf can be compiled into multiple .o files.
      
      This patch is the first step. It detects programs that contain
      multiple functions and checks that calls between them are valid.
      It splits the sequence of bpf instructions (one program) into a set
      of bpf functions that call each other. Calls to only known
      functions are allowed. Since all functions are presented to
      the verifier at once conceptually it is 'static linking'.
      
      Future plans:
      - introduce BPF_PROG_TYPE_LIBRARY and allow a set of bpf functions
        to be loaded into the kernel that can be later linked to other
        programs with concrete program types. Aka 'dynamic linking'.
      
      - introduce function pointer type and indirect calls to allow
        bpf functions call other dynamically loaded bpf functions while
        the caller bpf function is already executing. Aka 'runtime linking'.
        This will be more generic and more flexible alternative
        to bpf_tail_calls.
      
      FAQ:
      Q: Interpreter and JIT changes mean that new instruction is introduced ?
      A: No. The call instruction technically stays the same. Now it can call
         both kernel helpers and other bpf functions.
         Calling convention stays the same as well.
         From uapi point of view the call insn got new 'relocation' BPF_PSEUDO_CALL
         similar to BPF_PSEUDO_MAP_FD 'relocation' of bpf_ldimm64 insn.
      
      Q: What had to change on LLVM side?
      A: Trivial LLVM patch to allow calls was applied to upcoming 6.0 release:
         https://reviews.llvm.org/rL318614
         with few bugfixes as well.
         Make sure to build the latest llvm to have bpf_call support.
      
      More details in the patches.
      ====================
      Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      ef9fde06
    • D
      selftests/bpf: additional bpf_call tests · 28ab173e
      Daniel Borkmann 提交于
      Add some additional checks for few more corner cases.
      Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      28ab173e
    • A
      bpf: arm64: add JIT support for multi-function programs · db496944
      Alexei Starovoitov 提交于
      similar to x64 add support for bpf-to-bpf calls.
      When program has calls to in-kernel helpers the target call offset
      is known at JIT time and arm64 architecture needs 2 passes.
      With bpf-to-bpf calls the dynamically allocated function start
      is unknown until all functions of the program are JITed.
      Therefore (just like x64) arm64 JIT needs one extra pass over
      the program to emit correct call offsets.
      
      Implementation detail:
      Avoid being too clever in 64-bit immediate moves and
      always use 4 instructions (instead of 3-4 depending on the address)
      to make sure only one extra pass is needed.
      If some future optimization would make it worth while to optimize
      'call 64-bit imm' further, the JIT would need to do 4 passes
      over the program instead of 3 as in this patch.
      For typical bpf program address the mov needs 3 or 4 insns,
      so unconditional 4 insns to save extra pass is a worthy trade off
      at this state of JIT.
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      Acked-by: NDaniel Borkmann <daniel@iogearbox.net>
      Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      db496944
    • A
      bpf: x64: add JIT support for multi-function programs · 1c2a088a
      Alexei Starovoitov 提交于
      Typical JIT does several passes over bpf instructions to
      compute total size and relative offsets of jumps and calls.
      With multitple bpf functions calling each other all relative calls
      will have invalid offsets intially therefore we need to additional
      last pass over the program to emit calls with correct offsets.
      For example in case of three bpf functions:
      main:
        call foo
        call bpf_map_lookup
        exit
      foo:
        call bar
        exit
      bar:
        exit
      
      We will call bpf_int_jit_compile() indepedently for main(), foo() and bar()
      x64 JIT typically does 4-5 passes to converge.
      After these initial passes the image for these 3 functions
      will be good except call targets, since start addresses of
      foo() and bar() are unknown when we were JITing main()
      (note that call bpf_map_lookup will be resolved properly
      during initial passes).
      Once start addresses of 3 functions are known we patch
      call_insn->imm to point to right functions and call
      bpf_int_jit_compile() again which needs only one pass.
      Additional safety checks are done to make sure this
      last pass doesn't produce image that is larger or smaller
      than previous pass.
      
      When constant blinding is on it's applied to all functions
      at the first pass, since doing it once again at the last
      pass can change size of the JITed code.
      
      Tested on x64 and arm64 hw with JIT on/off, blinding on/off.
      x64 jits bpf-to-bpf calls correctly while arm64 falls back to interpreter.
      All other JITs that support normal BPF_CALL will behave the same way
      since bpf-to-bpf call is equivalent to bpf-to-kernel call from
      JITs point of view.
      Signed-off-by: NAlexei Starovoitov <ast@kernel.org>
      Acked-by: NDaniel Borkmann <daniel@iogearbox.net>
      Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>
      1c2a088a