Messages in this thread | | | Date | Thu, 30 Sep 2021 21:20:52 +0800 | From | kernel test robot <> | Subject | [gustavoars:for-next/cast-function 1/1] kernel/bpf/verifier.c:12470:16: warning: cast from 'unsigned int (*)(const void *, const struct bpf_insn *)' to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long |
| |
tree: https://git.kernel.org/pub/scm/linux/kernel/git/gustavoars/linux.git for-next/cast-function head: ffea83dd8823401f3922276d36aca35c3e99d7f3 commit: ffea83dd8823401f3922276d36aca35c3e99d7f3 [1/1] Makefile: Enable -Wcast-function-type config: i386-randconfig-r016-20210930 (attached as .config) compiler: clang version 14.0.0 (https://github.com/llvm/llvm-project 28981015526f2192440c18f18e8a20cd11b0779c) reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # https://git.kernel.org/pub/scm/linux/kernel/git/gustavoars/linux.git/commit/?id=ffea83dd8823401f3922276d36aca35c3e99d7f3 git remote add gustavoars https://git.kernel.org/pub/scm/linux/kernel/git/gustavoars/linux.git git fetch --no-tags gustavoars for-next/cast-function git checkout ffea83dd8823401f3922276d36aca35c3e99d7f3 # save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 ARCH=i386
If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> kernel/bpf/verifier.c:12470:16: warning: cast from 'unsigned int (*)(const void *, const struct bpf_insn *)' to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned long long)') converts to incompatible function type [-Wcast-function-type] insn->imm = BPF_CAST_CALL(func[subprog]->bpf_func) - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/filter.h:366:4: note: expanded from macro 'BPF_CAST_CALL' ((u64 (*)(u64, u64, u64, u64, u64))(x)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> kernel/bpf/verifier.c:12951:17: warning: cast from 'void *(*)(struct bpf_map *, void *)' to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned long long)') converts to incompatible function type [-Wcast-function-type] insn->imm = BPF_CAST_CALL(ops->map_lookup_elem) - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/filter.h:366:4: note: expanded from macro 'BPF_CAST_CALL' ((u64 (*)(u64, u64, u64, u64, u64))(x)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> kernel/bpf/verifier.c:12955:17: warning: cast from 'int (*)(struct bpf_map *, void *, void *, u64)' (aka 'int (*)(struct bpf_map *, void *, void *, unsigned long long)') to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned long long)') converts to incompatible function type [-Wcast-function-type] insn->imm = BPF_CAST_CALL(ops->map_update_elem) - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/filter.h:366:4: note: expanded from macro 'BPF_CAST_CALL' ((u64 (*)(u64, u64, u64, u64, u64))(x)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> kernel/bpf/verifier.c:12959:17: warning: cast from 'int (*)(struct bpf_map *, void *)' to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned long long)') converts to incompatible function type [-Wcast-function-type] insn->imm = BPF_CAST_CALL(ops->map_delete_elem) - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/filter.h:366:4: note: expanded from macro 'BPF_CAST_CALL' ((u64 (*)(u64, u64, u64, u64, u64))(x)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> kernel/bpf/verifier.c:12963:17: warning: cast from 'int (*)(struct bpf_map *, void *, u64)' (aka 'int (*)(struct bpf_map *, void *, unsigned long long)') to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned long long)') converts to incompatible function type [-Wcast-function-type] insn->imm = BPF_CAST_CALL(ops->map_push_elem) - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/filter.h:366:4: note: expanded from macro 'BPF_CAST_CALL' ((u64 (*)(u64, u64, u64, u64, u64))(x)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ kernel/bpf/verifier.c:12967:17: warning: cast from 'int (*)(struct bpf_map *, void *)' to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned long long)') converts to incompatible function type [-Wcast-function-type] insn->imm = BPF_CAST_CALL(ops->map_pop_elem) - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/filter.h:366:4: note: expanded from macro 'BPF_CAST_CALL' ((u64 (*)(u64, u64, u64, u64, u64))(x)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ kernel/bpf/verifier.c:12971:17: warning: cast from 'int (*)(struct bpf_map *, void *)' to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned long long)') converts to incompatible function type [-Wcast-function-type] insn->imm = BPF_CAST_CALL(ops->map_peek_elem) - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/filter.h:366:4: note: expanded from macro 'BPF_CAST_CALL' ((u64 (*)(u64, u64, u64, u64, u64))(x)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> kernel/bpf/verifier.c:12975:17: warning: cast from 'int (*)(struct bpf_map *, u32, u64)' (aka 'int (*)(struct bpf_map *, unsigned int, unsigned long long)') to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned long long)') converts to incompatible function type [-Wcast-function-type] insn->imm = BPF_CAST_CALL(ops->map_redirect) - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/filter.h:366:4: note: expanded from macro 'BPF_CAST_CALL' ((u64 (*)(u64, u64, u64, u64, u64))(x)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 8 warnings generated. -- >> kernel/bpf/hashtab.c:671:26: warning: cast from 'void *(*)(struct bpf_map *, void *)' to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned long long)') converts to incompatible function type [-Wcast-function-type] *insn++ = BPF_EMIT_CALL(BPF_CAST_CALL(__htab_map_lookup_elem)); ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/filter.h:366:4: note: expanded from macro 'BPF_CAST_CALL' ((u64 (*)(u64, u64, u64, u64, u64))(x)) ^ include/linux/filter.h:374:14: note: expanded from macro 'BPF_EMIT_CALL' .imm = ((FUNC) - __bpf_call_base) }) ^~~~ kernel/bpf/hashtab.c:712:26: warning: cast from 'void *(*)(struct bpf_map *, void *)' to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned long long)') converts to incompatible function type [-Wcast-function-type] *insn++ = BPF_EMIT_CALL(BPF_CAST_CALL(__htab_map_lookup_elem)); ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/filter.h:366:4: note: expanded from macro 'BPF_CAST_CALL' ((u64 (*)(u64, u64, u64, u64, u64))(x)) ^ include/linux/filter.h:374:14: note: expanded from macro 'BPF_EMIT_CALL' .imm = ((FUNC) - __bpf_call_base) }) ^~~~ kernel/bpf/hashtab.c:2400:26: warning: cast from 'void *(*)(struct bpf_map *, void *)' to 'u64 (*)(u64, u64, u64, u64, u64)' (aka 'unsigned long long (*)(unsigned long long, unsigned long long, unsigned long long, unsigned long long, unsigned long long)') converts to incompatible function type [-Wcast-function-type] *insn++ = BPF_EMIT_CALL(BPF_CAST_CALL(__htab_map_lookup_elem)); ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/filter.h:366:4: note: expanded from macro 'BPF_CAST_CALL' ((u64 (*)(u64, u64, u64, u64, u64))(x)) ^ include/linux/filter.h:374:14: note: expanded from macro 'BPF_EMIT_CALL' .imm = ((FUNC) - __bpf_call_base) }) ^~~~ 3 warnings generated. -- >> drivers/firewire/core-cdev.c:985:8: warning: cast from 'void (*)(struct fw_iso_context *, dma_addr_t, void *)' (aka 'void (*)(struct fw_iso_context *, unsigned int, void *)') to 'fw_iso_callback_t' (aka 'void (*)(struct fw_iso_context *, unsigned int, unsigned int, void *, void *)') converts to incompatible function type [-Wcast-function-type] cb = (fw_iso_callback_t)iso_mc_callback; ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1 warning generated.
vim +12470 kernel/bpf/verifier.c
9bac3d6d548e5c Alexei Starovoitov 2015-03-13 12337 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12338 static int jit_subprogs(struct bpf_verifier_env *env) 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12339 { 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12340 struct bpf_prog *prog = env->prog, **func, *tmp; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12341 int i, j, subprog_start, subprog_end = 0, len, subprog; a748c6975dea32 Maciej Fijalkowski 2020-09-16 12342 struct bpf_map *map_ptr; 7105e828c087de Daniel Borkmann 2017-12-20 12343 struct bpf_insn *insn; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12344 void *old_bpf_func; c4c0bdc0d2d084 Yonghong Song 2020-06-23 12345 int err, num_exentries; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12346 f910cefa32b6cd Jiong Wang 2018-05-02 12347 if (env->subprog_cnt <= 1) 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12348 return 0; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12349 7105e828c087de Daniel Borkmann 2017-12-20 12350 for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) { 69c087ba6225b5 Yonghong Song 2021-02-26 12351 if (bpf_pseudo_func(insn)) { 69c087ba6225b5 Yonghong Song 2021-02-26 12352 env->insn_aux_data[i].call_imm = insn->imm; 69c087ba6225b5 Yonghong Song 2021-02-26 12353 /* subprog is encoded in insn[1].imm */ 69c087ba6225b5 Yonghong Song 2021-02-26 12354 continue; 69c087ba6225b5 Yonghong Song 2021-02-26 12355 } 69c087ba6225b5 Yonghong Song 2021-02-26 12356 23a2d70c7a2f28 Yonghong Song 2021-02-04 12357 if (!bpf_pseudo_call(insn)) 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12358 continue; c7a897843224a9 Daniel Borkmann 2018-07-12 12359 /* Upon error here we cannot fall back to interpreter but c7a897843224a9 Daniel Borkmann 2018-07-12 12360 * need a hard reject of the program. Thus -EFAULT is c7a897843224a9 Daniel Borkmann 2018-07-12 12361 * propagated in any case. c7a897843224a9 Daniel Borkmann 2018-07-12 12362 */ 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12363 subprog = find_subprog(env, i + insn->imm + 1); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12364 if (subprog < 0) { 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12365 WARN_ONCE(1, "verifier bug. No program starts at insn %d\n", 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12366 i + insn->imm + 1); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12367 return -EFAULT; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12368 } 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12369 /* temporarily remember subprog id inside insn instead of 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12370 * aux_data, since next loop will split up all insns into funcs 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12371 */ f910cefa32b6cd Jiong Wang 2018-05-02 12372 insn->off = subprog; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12373 /* remember original imm in case JIT fails and fallback 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12374 * to interpreter will be needed 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12375 */ 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12376 env->insn_aux_data[i].call_imm = insn->imm; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12377 /* point imm to __bpf_call_base+1 from JITs point of view */ 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12378 insn->imm = 1; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12379 } 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12380 c454a46b5efd8e Martin KaFai Lau 2018-12-07 12381 err = bpf_prog_alloc_jited_linfo(prog); c454a46b5efd8e Martin KaFai Lau 2018-12-07 12382 if (err) c454a46b5efd8e Martin KaFai Lau 2018-12-07 12383 goto out_undo_insn; c454a46b5efd8e Martin KaFai Lau 2018-12-07 12384 c454a46b5efd8e Martin KaFai Lau 2018-12-07 12385 err = -ENOMEM; 6396bb221514d2 Kees Cook 2018-06-12 12386 func = kcalloc(env->subprog_cnt, sizeof(prog), GFP_KERNEL); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12387 if (!func) c7a897843224a9 Daniel Borkmann 2018-07-12 12388 goto out_undo_insn; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12389 f910cefa32b6cd Jiong Wang 2018-05-02 12390 for (i = 0; i < env->subprog_cnt; i++) { 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12391 subprog_start = subprog_end; 9c8105bd440223 Jiong Wang 2018-05-02 12392 subprog_end = env->subprog_info[i + 1].start; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12393 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12394 len = subprog_end - subprog_start; fb7dd8bca0139f Andrii Nakryiko 2021-08-15 12395 /* bpf_prog_run() doesn't call subprogs directly, 492ecee892c2a4 Alexei Starovoitov 2019-02-25 12396 * hence main prog stats include the runtime of subprogs. 492ecee892c2a4 Alexei Starovoitov 2019-02-25 12397 * subprogs don't have IDs and not reachable via prog_get_next_id 700d4796ef59f5 Alexei Starovoitov 2021-02-09 12398 * func[i]->stats will never be accessed and stays NULL 492ecee892c2a4 Alexei Starovoitov 2019-02-25 12399 */ 492ecee892c2a4 Alexei Starovoitov 2019-02-25 12400 func[i] = bpf_prog_alloc_no_stats(bpf_prog_size(len), GFP_USER); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12401 if (!func[i]) 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12402 goto out_free; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12403 memcpy(func[i]->insnsi, &prog->insnsi[subprog_start], 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12404 len * sizeof(struct bpf_insn)); 4f74d80971bce9 Daniel Borkmann 2017-12-20 12405 func[i]->type = prog->type; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12406 func[i]->len = len; 4f74d80971bce9 Daniel Borkmann 2017-12-20 12407 if (bpf_prog_calc_tag(func[i])) 4f74d80971bce9 Daniel Borkmann 2017-12-20 12408 goto out_free; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12409 func[i]->is_func = 1; ba64e7d8525236 Yonghong Song 2018-11-24 12410 func[i]->aux->func_idx = i; f263a81451c12d John Fastabend 2021-07-07 12411 /* Below members will be freed only at prog->aux */ ba64e7d8525236 Yonghong Song 2018-11-24 12412 func[i]->aux->btf = prog->aux->btf; ba64e7d8525236 Yonghong Song 2018-11-24 12413 func[i]->aux->func_info = prog->aux->func_info; f263a81451c12d John Fastabend 2021-07-07 12414 func[i]->aux->poke_tab = prog->aux->poke_tab; f263a81451c12d John Fastabend 2021-07-07 12415 func[i]->aux->size_poke_tab = prog->aux->size_poke_tab; ba64e7d8525236 Yonghong Song 2018-11-24 12416 a748c6975dea32 Maciej Fijalkowski 2020-09-16 12417 for (j = 0; j < prog->aux->size_poke_tab; j++) { f263a81451c12d John Fastabend 2021-07-07 12418 struct bpf_jit_poke_descriptor *poke; a748c6975dea32 Maciej Fijalkowski 2020-09-16 12419 f263a81451c12d John Fastabend 2021-07-07 12420 poke = &prog->aux->poke_tab[j]; f263a81451c12d John Fastabend 2021-07-07 12421 if (poke->insn_idx < subprog_end && f263a81451c12d John Fastabend 2021-07-07 12422 poke->insn_idx >= subprog_start) f263a81451c12d John Fastabend 2021-07-07 12423 poke->aux = func[i]->aux; a748c6975dea32 Maciej Fijalkowski 2020-09-16 12424 } a748c6975dea32 Maciej Fijalkowski 2020-09-16 12425 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12426 /* Use bpf_prog_F_tag to indicate functions in stack traces. 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12427 * Long term would need debug info to populate names 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12428 */ 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12429 func[i]->aux->name[0] = 'F'; 9c8105bd440223 Jiong Wang 2018-05-02 12430 func[i]->aux->stack_depth = env->subprog_info[i].stack_depth; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12431 func[i]->jit_requested = 1; e6ac2450d6dee3 Martin KaFai Lau 2021-03-24 12432 func[i]->aux->kfunc_tab = prog->aux->kfunc_tab; c454a46b5efd8e Martin KaFai Lau 2018-12-07 12433 func[i]->aux->linfo = prog->aux->linfo; c454a46b5efd8e Martin KaFai Lau 2018-12-07 12434 func[i]->aux->nr_linfo = prog->aux->nr_linfo; c454a46b5efd8e Martin KaFai Lau 2018-12-07 12435 func[i]->aux->jited_linfo = prog->aux->jited_linfo; c454a46b5efd8e Martin KaFai Lau 2018-12-07 12436 func[i]->aux->linfo_idx = env->subprog_info[i].linfo_idx; c4c0bdc0d2d084 Yonghong Song 2020-06-23 12437 num_exentries = 0; c4c0bdc0d2d084 Yonghong Song 2020-06-23 12438 insn = func[i]->insnsi; c4c0bdc0d2d084 Yonghong Song 2020-06-23 12439 for (j = 0; j < func[i]->len; j++, insn++) { c4c0bdc0d2d084 Yonghong Song 2020-06-23 12440 if (BPF_CLASS(insn->code) == BPF_LDX && c4c0bdc0d2d084 Yonghong Song 2020-06-23 12441 BPF_MODE(insn->code) == BPF_PROBE_MEM) c4c0bdc0d2d084 Yonghong Song 2020-06-23 12442 num_exentries++; c4c0bdc0d2d084 Yonghong Song 2020-06-23 12443 } c4c0bdc0d2d084 Yonghong Song 2020-06-23 12444 func[i]->aux->num_exentries = num_exentries; ebf7d1f508a738 Maciej Fijalkowski 2020-09-16 12445 func[i]->aux->tail_call_reachable = env->subprog_info[i].tail_call_reachable; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12446 func[i] = bpf_int_jit_compile(func[i]); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12447 if (!func[i]->jited) { 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12448 err = -ENOTSUPP; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12449 goto out_free; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12450 } 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12451 cond_resched(); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12452 } a748c6975dea32 Maciej Fijalkowski 2020-09-16 12453 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12454 /* at this point all bpf functions were successfully JITed 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12455 * now populate all bpf_calls with correct addresses and 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12456 * run last pass of JIT 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12457 */ f910cefa32b6cd Jiong Wang 2018-05-02 12458 for (i = 0; i < env->subprog_cnt; i++) { 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12459 insn = func[i]->insnsi; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12460 for (j = 0; j < func[i]->len; j++, insn++) { 69c087ba6225b5 Yonghong Song 2021-02-26 12461 if (bpf_pseudo_func(insn)) { 69c087ba6225b5 Yonghong Song 2021-02-26 12462 subprog = insn[1].imm; 69c087ba6225b5 Yonghong Song 2021-02-26 12463 insn[0].imm = (u32)(long)func[subprog]->bpf_func; 69c087ba6225b5 Yonghong Song 2021-02-26 12464 insn[1].imm = ((u64)(long)func[subprog]->bpf_func) >> 32; 69c087ba6225b5 Yonghong Song 2021-02-26 12465 continue; 69c087ba6225b5 Yonghong Song 2021-02-26 12466 } 23a2d70c7a2f28 Yonghong Song 2021-02-04 12467 if (!bpf_pseudo_call(insn)) 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12468 continue; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12469 subprog = insn->off; 0d306c31b2f773 Prashant Bhole 2019-04-16 @12470 insn->imm = BPF_CAST_CALL(func[subprog]->bpf_func) - 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12471 __bpf_call_base; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12472 } 2162fed49fa86c Sandipan Das 2018-05-24 12473 2162fed49fa86c Sandipan Das 2018-05-24 12474 /* we use the aux data to keep a list of the start addresses 2162fed49fa86c Sandipan Das 2018-05-24 12475 * of the JITed images for each function in the program 2162fed49fa86c Sandipan Das 2018-05-24 12476 * 2162fed49fa86c Sandipan Das 2018-05-24 12477 * for some architectures, such as powerpc64, the imm field 2162fed49fa86c Sandipan Das 2018-05-24 12478 * might not be large enough to hold the offset of the start 2162fed49fa86c Sandipan Das 2018-05-24 12479 * address of the callee's JITed image from __bpf_call_base 2162fed49fa86c Sandipan Das 2018-05-24 12480 * 2162fed49fa86c Sandipan Das 2018-05-24 12481 * in such cases, we can lookup the start address of a callee 2162fed49fa86c Sandipan Das 2018-05-24 12482 * by using its subprog id, available from the off field of 2162fed49fa86c Sandipan Das 2018-05-24 12483 * the call instruction, as an index for this list 2162fed49fa86c Sandipan Das 2018-05-24 12484 */ 2162fed49fa86c Sandipan Das 2018-05-24 12485 func[i]->aux->func = func; 2162fed49fa86c Sandipan Das 2018-05-24 12486 func[i]->aux->func_cnt = env->subprog_cnt; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12487 } f910cefa32b6cd Jiong Wang 2018-05-02 12488 for (i = 0; i < env->subprog_cnt; i++) { 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12489 old_bpf_func = func[i]->bpf_func; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12490 tmp = bpf_int_jit_compile(func[i]); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12491 if (tmp != func[i] || func[i]->bpf_func != old_bpf_func) { 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12492 verbose(env, "JIT doesn't support bpf-to-bpf calls\n"); c7a897843224a9 Daniel Borkmann 2018-07-12 12493 err = -ENOTSUPP; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12494 goto out_free; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12495 } 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12496 cond_resched(); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12497 } 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12498 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12499 /* finally lock prog and jit images for all functions and 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12500 * populate kallsysm 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12501 */ f910cefa32b6cd Jiong Wang 2018-05-02 12502 for (i = 0; i < env->subprog_cnt; i++) { 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12503 bpf_prog_lock_ro(func[i]); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12504 bpf_prog_kallsyms_add(func[i]); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12505 } 7105e828c087de Daniel Borkmann 2017-12-20 12506 7105e828c087de Daniel Borkmann 2017-12-20 12507 /* Last step: make now unused interpreter insns from main 7105e828c087de Daniel Borkmann 2017-12-20 12508 * prog consistent for later dump requests, so they can 7105e828c087de Daniel Borkmann 2017-12-20 12509 * later look the same as if they were interpreted only. 7105e828c087de Daniel Borkmann 2017-12-20 12510 */ 7105e828c087de Daniel Borkmann 2017-12-20 12511 for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) { 69c087ba6225b5 Yonghong Song 2021-02-26 12512 if (bpf_pseudo_func(insn)) { 69c087ba6225b5 Yonghong Song 2021-02-26 12513 insn[0].imm = env->insn_aux_data[i].call_imm; 69c087ba6225b5 Yonghong Song 2021-02-26 12514 insn[1].imm = find_subprog(env, i + insn[0].imm + 1); 69c087ba6225b5 Yonghong Song 2021-02-26 12515 continue; 69c087ba6225b5 Yonghong Song 2021-02-26 12516 } 23a2d70c7a2f28 Yonghong Song 2021-02-04 12517 if (!bpf_pseudo_call(insn)) 7105e828c087de Daniel Borkmann 2017-12-20 12518 continue; 7105e828c087de Daniel Borkmann 2017-12-20 12519 insn->off = env->insn_aux_data[i].call_imm; 7105e828c087de Daniel Borkmann 2017-12-20 12520 subprog = find_subprog(env, i + insn->off + 1); dbecd7388476ae Sandipan Das 2018-05-24 12521 insn->imm = subprog; 7105e828c087de Daniel Borkmann 2017-12-20 12522 } 7105e828c087de Daniel Borkmann 2017-12-20 12523 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12524 prog->jited = 1; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12525 prog->bpf_func = func[0]->bpf_func; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12526 prog->aux->func = func; f910cefa32b6cd Jiong Wang 2018-05-02 12527 prog->aux->func_cnt = env->subprog_cnt; e16301fbe1837c Martin KaFai Lau 2021-03-24 12528 bpf_prog_jit_attempt_done(prog); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12529 return 0; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12530 out_free: f263a81451c12d John Fastabend 2021-07-07 12531 /* We failed JIT'ing, so at this point we need to unregister poke f263a81451c12d John Fastabend 2021-07-07 12532 * descriptors from subprogs, so that kernel is not attempting to f263a81451c12d John Fastabend 2021-07-07 12533 * patch it anymore as we're freeing the subprog JIT memory. f263a81451c12d John Fastabend 2021-07-07 12534 */ f263a81451c12d John Fastabend 2021-07-07 12535 for (i = 0; i < prog->aux->size_poke_tab; i++) { f263a81451c12d John Fastabend 2021-07-07 12536 map_ptr = prog->aux->poke_tab[i].tail_call.map; f263a81451c12d John Fastabend 2021-07-07 12537 map_ptr->ops->map_poke_untrack(map_ptr, prog->aux); f263a81451c12d John Fastabend 2021-07-07 12538 } f263a81451c12d John Fastabend 2021-07-07 12539 /* At this point we're guaranteed that poke descriptors are not f263a81451c12d John Fastabend 2021-07-07 12540 * live anymore. We can just unlink its descriptor table as it's f263a81451c12d John Fastabend 2021-07-07 12541 * released with the main prog. f263a81451c12d John Fastabend 2021-07-07 12542 */ a748c6975dea32 Maciej Fijalkowski 2020-09-16 12543 for (i = 0; i < env->subprog_cnt; i++) { a748c6975dea32 Maciej Fijalkowski 2020-09-16 12544 if (!func[i]) a748c6975dea32 Maciej Fijalkowski 2020-09-16 12545 continue; f263a81451c12d John Fastabend 2021-07-07 12546 func[i]->aux->poke_tab = NULL; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12547 bpf_jit_free(func[i]); a748c6975dea32 Maciej Fijalkowski 2020-09-16 12548 } 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12549 kfree(func); c7a897843224a9 Daniel Borkmann 2018-07-12 12550 out_undo_insn: 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12551 /* cleanup main prog to be interpreted */ 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12552 prog->jit_requested = 0; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12553 for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) { 23a2d70c7a2f28 Yonghong Song 2021-02-04 12554 if (!bpf_pseudo_call(insn)) 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12555 continue; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12556 insn->off = 0; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12557 insn->imm = env->insn_aux_data[i].call_imm; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12558 } e16301fbe1837c Martin KaFai Lau 2021-03-24 12559 bpf_prog_jit_attempt_done(prog); 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12560 return err; 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12561 } 1c2a088a6626d4 Alexei Starovoitov 2017-12-14 12562
:::::: The code at line 12470 was first introduced by commit :::::: 0d306c31b2f77391dacdeaad4470c577f2aecc4f bpf: use BPF_CAST_CALL for casting bpf call
:::::: TO: Prashant Bhole <bhole_prashant_q7@lab.ntt.co.jp> :::::: CC: Alexei Starovoitov <ast@kernel.org>
--- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org [unhandled content-type:application/gzip] | |