1. For architecture with overlapping address space, error out when
   bpf_probe_read_user is not available.
2. For arch with non overlapping address space, if bpf_probe_read_user
   is not available bpf_probe_read_user is implicitly converted to
   bpf_probe_read.
3. Use bpf_probe_read_kernel instead of bpf_probe_read. When
   bpf_probe_read_kernel is not available, fallback to bpf_probe_read.
   If bpf_probe_read is not available, then bcc would fail anyways.
4. See kernel commit 0ebeea8ca8a4 ("bpf: Restrict bpf_probe_read{,
   str}() only to archs where they work")

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Acked-by: Ilya Leoshkevich <iii@linux.ibm.com>

`sched_process_exit` tracepoint is called when thread terminates.
So exitsnoop shows line per each thread termination if the process
is multi-thread process. This is not useful when people wants to
know why process terminates, not thread.

So this changes exitsnoop default behavior which traces process termination
instead of thread termination. And add `--per-thread` option which behaves
as original exitsnoop implementation.

(as title)

In previous patches, I added the option --cgroupmap to filter events
belonging to a set of cgroup-v2. Although this approach works fine with
systemd services and containers when cgroup-v2 is enabled, it does not
work with containers when only cgroup-v1 is enabled because
bpf_get_current_cgroup_id() only works with cgroup-v2. It also requires
Linux 4.18 to get this bpf helper function.

This patch adds an additional way to filter by containers, using mount
namespaces.

Note that this does not help with systemd services since they normally
don't create a new mount namespace (unless you set some options like
'ReadOnlyPaths=', see "man 5 systemd.exec").

My goal with this patch is to filter Kubernetes pods, even on
distributions with an older kernel (<4.18) or without cgroup-v2 enabled.

- This is only implemented for tools that already support filtering by
  cgroup id (bindsnoop, capable, execsnoop, profile, tcpaccept, tcpconnect,
  tcptop and tcptracer).

- I picked the mount namespace because the other namespaces could be
  disabled in Kubernetes (e.g. HostNetwork, HostPID, HostIPC).

It can be tested by following the example in docs/special_filtering added
in this commit, to avoid compiling locally the following command can be used

```
sudo bpftool map create /sys/fs/bpf/mnt_ns_set type hash key 8 value 4 \
  entries 128 name mnt_ns_set flags 0
docker run -ti --rm --privileged \
  -v /usr/src:/usr/src -v /lib/modules:/lib/modules \
  -v /sys/fs/bpf:/sys/fs/bpf --pid=host kinvolk/bcc:alban-containers-filters \
  /usr/share/bcc/tools/execsnoop --mntnsmap /sys/fs/bpf/mnt_ns_set

```

Co-authored-by: Alban Crequy <alban@kinvolk.io>
Co-authored-by: Mauricio Vásquez <mauricio@kinvolk.io>

1ad2656a ("Add support_kfunc function to BPF object") added new functions
to libbcc-py but didn't set the restype and argstypes for those. It's causing
the `bpf_has_kernel_btf` function to return True in systems without support
for BTF, making tools like opensnoop, klockstat and any other using kfuncs
unusable in those systems.

The following Python script reproduces the problem:

```
from bcc import BPF
print(BPF.support_kfunc())
```

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>

The KFUNC_PROBE macro is using "void" as return type, this is causing problems
in some tools that have a filtering enable that returns 0.

Reproducer: (Notice that it requires BTF support)

```
$ python opensnoop.py --pid 5
/virtual/main.c:33:21: error: void function '____kretfunc__do_sys_open' should not return a value [-Wreturn-type]
    if (pid != 5) { return 0; }
                    ^      ~
1 error generated.
...
```

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>

Sync with latest libbpf repo. Update virtual_bpf.h,
helpers.h, docs, etc.

Signed-off-by: Yonghong Song <yhs@fb.com>

This tools is about reporting IOs per directory.
That's a clone of filetop but works in a different way :
- user specify a set of globs to select a list of directories to watch
- dirtop extracts the inode_id of the selected directories
- the bpf program receives the list of top directories to consider
- when vfs_{read|write} occurs, the bpf program check
  if one of the parents is part of the list we search for
- if it matches, the io is accounted

On the python side, the program will reconcilate IOs per directory and print stats.

While filetop list the programs and filename, dirtop only list the directory name.

A typical usages looks like :

	[root@host]: dirtop.py -d '/hdfs/uuid/*/yarn'
	14:56:33 loadavg: 52.21 48.81 37.78 53/2721 28720

	READS  WRITES R_Kb     W_Kb     PATH
	36821  7632   238219   149183   /hdfs/uuid/d04fccd8-bc72-4ed9-bda4-c5b6893f1405/yarn
	20823  2      196290   3        /hdfs/uuid/b94cbf3f-76b1-4ced-9043-02d450b9887c/yarn
	16059  12064  109748   85778    /hdfs/uuid/250b21c8-1714-45fe-8c08-d45d0271c6bd/yarn
	14128  20360  106287   81440    /hdfs/uuid/4a833770-767e-43b3-b696-dc98901bce26/yarn
	15883  4991   86014    82075    /hdfs/uuid/0cc3683f-4800-4c73-8075-8d77dc7cf116/yarn
	11182  4485   28834    116917   /hdfs/uuid/7d512fe7-b20d-464c-a75a-dbf8b687ee1c/yarn
	11848  7810   103139   31240    /hdfs/uuid/2c6a7223-cb18-4916-a1b6-8cd02bda1d31/yarn
	10418  1272   114842   18       /hdfs/uuid/76dc0b77-e2fd-4476-818f-2b5c3c452396/yarn
	10066  6630   93969    20218    /hdfs/uuid/c11da291-28de-4a77-873e-44bb452d238b/yarn
	13648  15453  39450    53744    /hdfs/uuid/99c178d5-a209-4af2-8467-7382c7f03c1b/yarn
	9509   2049   31363    48219    /hdfs/uuid/a78f846a-58c4-4d10-a9f5-42f16a6134a0/yarn
	8112   2178   13765    63479    /hdfs/uuid/bf829d08-1455-45b8-81fa-05c3303e8c45/yarn
	4327   0      37544    0        /hdfs/uuid/fada8004-53ff-48df-9396-165d8e42925b/yarn
	2238   2742   72       50       /hdfs/uuid/b3b2a2ed-f6c1-4641-86bf-2989dd932411/yarn
	3716   0      47       0        /hdfs/uuid/8138a53b-b942-44d3-82df-51575f1a3901/yarn

Signed-off-by: Erwan Velu <e.velu@criteo.com>

Co-authored-by: Erwan Velu <e.velu@criteo.com>

Github actions are failing because the docker image used for testing is based
in Ubuntu 19.04 that is EOL now. This commit downgrades the version used for
testing to 18.04 that is LTS. It also installs the "util-linux" package that
includes the "unshare" command.

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>

Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>

I use clang-8.0.1 and gcc-8.3.1 to compile bcc-0.8.1,
without -fPIC, will get errors like this:
/usr/bin/ld: CMakeFiles/test_libbcc.dir/test_libbcc.cc.o: relocation
R_X86_64_32S against symbol `_ZTVN5Catch21LegacyReporterAdapterE' can
not be used when making a PIE object; recompile with -fPIC
/usr/bin/ld: CMakeFiles/test_libbcc.dir/test_c_api.cc.o: relocation
R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE
object; recompile with -fPIC
/usr/bin/ld: CMakeFiles/CGroupTest.dir/CGroupTest.cc.o: relocation
R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE
object; recompile with -fPIC

Signed-off-by: Chunmei Xu <xuchunmei@linux.alibaba.com>

add nfs v3 support to nfsdist.py

Co-authored-by: sabbene <sabbene@sabbene.nvidia.com>

Bcc tools fixe for trace.py and argdist.py for bpf_probe_read_user.

Make it clear which file the USDT runtime files to find and suggest a
fix.

* Add perf event data collection example for an userspace application
* Add comments for potential issues in perf_event example

bool returns True for negative integers. Hence bcc tools
tries to switch to kfunc instead of kprobes, even when the
btf data is not found.

For libbpf_find_vmlinux_btf_id, When err <= 0 , btf is not found.

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Acked-by: Ilya Leoshkevich <iii@linux.ibm.com>

Arguments of a probe point can be either user pointer or kernel
pointer.

Previously:
- tools/trace.py 'do_sys_open "%s", arg2'
  When reading arg2 as char *, it would resolve to bpf_probe_read.

Now:
- tools/trace.py 'do_sys_open "%s", arg2@user'
  - When reading arg2 as char *, it is resolved to bpf_probe_read_user.
- tools/trace.py 'do_sys_open (STRCMP("test.txt", arg2@user)) "%s", arg2'
  - For arg2 char * read, bpf_probe_read_user is utilized

To distinguish this, add arg@user.
- All userspace probes char *read converted to bpf_probe_read_user
- Syscall/kprobes with arg[1-6]@user attribute are converted to
  bpf_probe_read_user.

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Acked-by: Ilya Leoshkevich <iii@linux.ibm.com>

argdist traces probe functions and its parameter values.

Add functionality to convert:
- All userspace probes char * read to bpf_probe_read_user()
- Syscall/kprobes char* params with __user attribute to bpf_probe_read_user()

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Acked-by: Ilya Leoshkevich <iii@linux.ibm.com>

bcc tools like trace.py and argdist.py uses _generate_streq_function()
functions to convert char * read to bpf_probe_read/bpf_probe_read_user.
Refactor it and move the common functionality to utils.py.

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Acked-by: Ilya Leoshkevich <iii@linux.ibm.com>

bcc: Use bpf_probe_read_user in tools and provide backward compatibility

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>

This is essential for architecture which do have overlapping address space.
- bpf_probe_read_kernel() shall be used for reading data from kernel space
to the bpf vm.
- bpf_probe_read_user() shall be used for reading data from user space
  to the bpf vm.

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>

1. Commit fa697140f9a2 ("syscalls/x86: Use 'struct pt_regs' based
   syscall calling convention for 64-bit syscalls") changed the raw
   parameter passed to the syscall entry function from a list of
   parameters supplied in user space to a single `pt_regs *` parameter
   (ARCH_HAS_SYSCALL_WRAPPER)

2. But ARCH_HAS_SYSCALL_WRAPPER in s390 is not used for that purpose.
   See commit a18f03cd89e9 ("s390: autogenerate compat syscall
   wrappers")

3. Use direct parameter assignment assumption for s390 syscall probe
   instead.

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>

s390 has overlapping address space for user and kernel. Hence separation of
bpf_probe_read_user and bpf_probe_read_kernel is essential. Commit 6ae08ae3dea2
("bpf: Add probe_read_{user, kernel} and probe_read_{user, kernel}_str
helpers") introduced these changes into the kernel. However, bcc tools does not
respect it.

As a workaround, perform the following:
1. Use bpf_probe_read_user() explicitly in the bcc tools.
2. When kernel version < 5.5, perform the checks if the
   bpf_probe_read_user kernel helper is present in the backported kernel
   as well. If not found, then fallback from bpf_probe_read_user to
   bpf_probe_read.

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>

Earlier, it was assumed that ELF ABI v2 is used only on
little-endian powerpc64 environments but it seems this
ABI can be used independently of endianness.

It is expected that any C preprocessor that conforms to
the ELF ABI v2 specification must predefine the _CALL_ELF
macro and set its value to 2.

Instead of looking at __BYTE_ORDER__ to determine whether
to use the Local Entry Point (LEP) of symbols, one should
look at the _CALL_ELF macro instead as this is ABI-related.

Similarly, _CALL_ELF should be used only for determining
the ABI version and not the endianness.

Reported-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Fixes: bbd4180c ("Fix uprobes on powerpc64")
Fixes: 10869523 ("clang: Add support to build eBPF for user specified ARCH")
Acked-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Signed-off-by: Sandipan Das <sandipan@linux.ibm.com>

added changelog for release v0.14.0

sync to libbpf v0.0.8. Add newer helpers to
helpers.h, libbpf.c error reporting and docs.

The llvm CreateCall used in bcc is deprecated in llvm 11:
  https://reviews.llvm.org/D76269
The llvm CreateMemCpy is changed in llvm 11 as well:
  https://reviews.llvm.org/D71473

This caused bcc compilation error.

  /home/yhs/work/bcc/src/cc/frontends/b/codegen_llvm.cc: In member function
     ‘ebpf::StatusTuple ebpf::cc::CodegenLLVM::emit_log(ebpf::cc::Method CallExprNode*)’:
  /home/yhs/work/bcc/src/cc/frontends/b/codegen_llvm.cc:691:39: error: no matching function for call to
     ‘llvm::IRBuilder<>::CreateCall(llvm::Value*&, std::vector<llvm::Value*, std::allocator<llvm::Value*> >&)’
     expr_ = B.CreateCall(printk_fn, args);
                                       ^
  ...

  /home/yhs/work/bcc/src/cc/frontends/b/codegen_llvm.cc: In member function
     ‘virtual ebpf::StatusTuple ebpf::cc::CodegenLLVM::visit_string_exp_node(ebpf::cc::StringExprNode*)’:
  /home/yhs/work/bcc/src/cc/frontends/b/codegen_llvm.cc:440:55: error: no matching function for call to
     ‘llvm::IRBuilder<>::CreateMemCpy(llvm:Value*&, int, llvm::Value*&, int, std::__cxx11::basic_string<char>::size_type)’
   B.CreateMemCpy(ptr, 1, global, 1, n->val_.size() + 1);
                                                       ^
  ...

This patch fixed the compilation issue.

 add bcc/bcc_version.h to install

Added kernel recompile guidance for  libbpf CO-RE

Expand io_latencies example to iolatpcts tool

some distros already packaging clang 10 (checked fedora and arch) no longer
ship all the individual libclang*.so component libraries.
Instead, clang from 9.0 onwards provides a new lib, libclang-cpp.so, which
includes everything we need.

Tell cmake to use it if the individual libraries are no longer found.
(Build-wise, if both are present it is more efficient to use the individual
components so keep these first)

This is simplified version of tools/biolatpcts.py and demonstrates how to
obtain and calculate latency percentiles.