It is recommended to use bpf_probe_read_kernel_{str} in the bpf tools.
See kernel commit 0ebeea8ca8a4 ("bpf: Restrict bpf_probe_read{, str}()
only to archs where they work")

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Acked-by: Ilya Leoshkevich <iii@linux.ibm.com>

Arguments of a probe point can be either user pointer or kernel
pointer.

Previously:
- tools/trace.py 'do_sys_open "%s", arg2'
  When reading arg2 as char *, it would resolve to bpf_probe_read.

Now:
- tools/trace.py 'do_sys_open "%s", arg2@user'
  - When reading arg2 as char *, it is resolved to bpf_probe_read_user.
- tools/trace.py 'do_sys_open (STRCMP("test.txt", arg2@user)) "%s", arg2'
  - For arg2 char * read, bpf_probe_read_user is utilized

To distinguish this, add arg@user.
- All userspace probes char *read converted to bpf_probe_read_user
- Syscall/kprobes with arg[1-6]@user attribute are converted to
  bpf_probe_read_user.

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Acked-by: Ilya Leoshkevich <iii@linux.ibm.com>

In the normal develop, will produce many event on the same tracepoint,
like do_sys_open, a executable program will open many files but
developer only has interesting on the specific file. So this filter
will
help developer to get their interesting msg

Signed-off-by: tty5 <tty.fqq@gmail.com>

porting from opensnoop

Signed-off-by: tty5 <tty.fqq@gmail.com>

Do not use BPF_F_REUSE_STACKID if the stack id is used
together with process specific info like
pid/tgid/comm. Using BPF_F_REUSE_STACKID may cause
stack id pointing to a different stack later on.

Signed-off-by: Yonghong Song <yhs@fb.com>

This patch added cgroup based filtering in trace.py.

If a cgroup path is specified by the user, one cgroup
array map will be added to the program:
  BPF_CGROUP_ARRAY(__cgroup, 1);

Each probe will have a filter like below:
  if (__cgroup.check_current_task(0) <= 0) { return 0; }
to filter out any events not happening in the cgroup
hierarchy as specified by the user.

The trace.py updated the `__cgroup` map with user provided
cgroup path information before attaching bpf functions
to events for probe function(s).

An example like below:
  $ trace.py -v -c /sys/fs/cgroup/system.slice/workload.service \
    '__x64_sys_nanosleep' '__x64_sys_clone'
     PID     TID     COMM            FUNC
     3191578 3191583 BaseAgentEvents __x64_sys_nanosleep
     3191578 3191579 FutureTimekeepr __x64_sys_clone
     3191578 3191583 BaseAgentEvents __x64_sys_nanosleep
     3191578 3191583 BaseAgentEvents __x64_sys_nanosleep
since workload.service only contains one process 3191578.

Going up the hierarchy to system.slice will have more processes
and hence more results:
  $ trace.py -v -c /sys/fs/cgroup/system.slice \
    '__x64_sys_nanosleep' '__x64_sys_clone'
     PID     TID     COMM            FUNC
     591542  591677  dynoScribe      __x64_sys_nanosleep
     591610  591613  mcreplay2       __x64_sys_nanosleep
     553252  553252  sleeperagent    __x64_sys_nanosleep
     591610  591613  mcreplay2       __x64_sys_nanosleep
     553252  553252  sleeperagent    __x64_sys_nanosleep

Signed-off-by: Yonghong Song <yhs@fb.com>

New command line options have been added to tools/trace.py to support the 
new BUILD_ID stackmap. List of symbol files can be added to the script to 
resolve symbols from build id as reported by the kernel in the stack trace
Updated man page and added an example usage

On x64 from kernel v4.17 onwards, a indirect table is used to
rewrite syscall parameters in trace.py. However, it only works
for arg1. This patch fixes it for arg2-arg6 too.

Fixes: 2da34267 ("generate indirect parameter assignment if arch uses syscall wrapper (#1816)")
Signed-off-by: Xiaozhou Liu <liuxiaozhou@bytedance.com>

Compiler shows warning "incompatible integer to pointer conversion
initializing" while compiling bpf program.
This patch adds necessary typecast when assigning PT_REGS_PARAM vaules
to struct pt_regs pointer

* tools: uses 'replace' error handler by default in decode()

Tools might encouter characters from non utf-8 charset (e.g. a file
name). When this happen, it's better to replace the unexpected
character by a question mark than crash the tool when all we do is
to print the string.

* tools: fix a bytes/string issue in attach_perf_event()

Currently, trace.py does not support "long" and "unsigned long"
types and it often caught users with a surprise and they are
not sure what is the problem. For example, for kernel function:
  void blk_mq_delay_kick_requeue_list(struct request_queue *q, unsigned long msecs)
The following
  $ sudo ./trace.py 'blk_mq_delay_kick_requeue_list(void *notused, unsigned long msecs) "msecs = %lu", msecs'
  list index out of range

With this patch,
  $ sudo ./trace.py 'blk_mq_delay_kick_requeue_list(void *notused, unsigned long msecs) "msecs = %lu", msecs'
  PID     TID     COMM            FUNC             -
  ^C
  $ sudo ./trace.py 'blk_mq_delay_kick_requeue_list(void *notused, unsigned long msecs) "msecs = %ld", msecs'
  PID     TID     COMM            FUNC             -
  ^C
  $ sudo ./trace.py 'blk_mq_delay_kick_requeue_list(void *notused, unsigned long msecs) "msecs = %lx", msecs'
  PID     TID     COMM            FUNC             -
  ^C
  $

Signed-off-by: Yonghong Song <yhs@fb.com>

Currently, trace.py failed for the following command:
  $ sudo ./trace.py 'filename_lookup(int dfd, struct filename *name) "%s", name->name'
  ...
  0: (bf) r6 = r1
  1: (79) r7 = *(u64 *)(r6 +104)
  ...
  32: (15) if r1 == 0x0 goto pc+5
  R0=inv(id=0) R1=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=inv(id=0)
  R8=inv0 R10=fp0,call_-1 fp-8=0 fp-16=0 fp-24=0 fp-32=0 fp-40=0 fp-48=0 fp-56=0 fp-64=0 fp-72=0 fp-80=0
  33: (79) r3 = *(u64 *)(r7 +0)
  R7 invalid mem access 'inv'

For string format argument, the trace.py generates the below code:
        if (name->name != 0) {
                bpf_probe_read(&__data.v0, sizeof(__data.v0), (void *)name->name);
        }
Right now, bcc skips the rewriter for the third argument of bpf_probe_read to avoid
unnecessary nested bpf_probe_read and other potential issues.
This causes name->name memory access not transformed with bpf_probe_read and hence
the verifier complains.

To fix the issue, this patch did the following transformation using an
temporary variable to hold the src address:
        if (name->name != 0) {
                void *__tmp = (void *)name->name;
                bpf_probe_read(&__data.v0, sizeof(__data.v0), __tmp);
        }
This way, rewriter can do the work properly.

Signed-off-by: Yonghong Song <yhs@fb.com>

* [trace.py]: allow to use STRCMP helper with binary values

Summary:
sometimes in probe you want to compare char* w/ some predefined value
which is not a string. e.g. setsockopt syscall has signature like this:
sys_setsockopt(int fd, int level, int optname, char* optval, int optlen)
and if you want to catch where/who is setting up specific value you are
forced to compare optval against some predefined array. it's not
possible today w/ trace.py and in this diff i'm adding such ability

Test Plan:
as example: we want to catch setsockopt when someone is setting up
IP_TOS equal to 108
trace.py 'sys_setsockopt(int fd, int level, int optname, char* optval,
int optlen)(level==0 && optname == 1 && STRCMP("{0x6C,0x00, 0x00,
0x00}", optval))' -U -M 1 --bin_cmp -v

without this new modifier:
static inline bool streq_0(char const *ignored, uintptr_t str) {
        char needle[] = "{0x6C,0x00, 0x00, 0x00}";
        char haystack[sizeof(needle)];
        bpf_probe_read(&haystack, sizeof(haystack), (void *)str);
        for (int i = 0; i < sizeof(needle) - 1; ++i) {
                if (needle[i] != haystack[i]) {
                        return false;
                }
        }
        return true;
}

// see needle is qouted above

with:

tatic inline bool streq_0(char const *ignored, uintptr_t str) {
        char needle[] = {0x6C,0x00, 0x00, 0x00};
        char haystack[sizeof(needle)];
        bpf_probe_read(&haystack, sizeof(haystack), (void *)str);
        for (int i = 0; i < sizeof(needle) - 1; ++i) {
                if (needle[i] != haystack[i]) {
                        return false;
                }
        }
        return true;
}

...
PID     TID     COMM            FUNC             -
1855611 1863183 worker          sys_setsockopt   found

* adding example of --bin_cmp flag usage

Fix issue #1802.

On x64, the following commit (in 4.17) changed the raw parameter passed to
the syscall entry function from a list of parameters supplied in user space
to a single `pt_regs *` parameter. Also in 4.17, x64 syscall entry function
is changed from `sys_<name>` to `__x64_sys_<name>`.

```
commit fa697140f9a20119a9ec8fd7460cc4314fbdaff3
Author: Dominik Brodowski <linux@dominikbrodowski.net>
Date:   Thu Apr 5 11:53:02 2018 +0200

    syscalls/x86: Use 'struct pt_regs' based syscall calling convention for 64-bit syscalls

    Let's make use of ARCH_HAS_SYSCALL_WRAPPER=y on pure 64-bit x86-64 systems:

    Each syscall defines a stub which takes struct pt_regs as its only
    argument. It decodes just those parameters it needs, e.g:

            asmlinkage long sys_xyzzy(const struct pt_regs *regs)
            {
                    return SyS_xyzzy(regs->di, regs->si, regs->dx);
            }

    This approach avoids leaking random user-provided register content down
    the call chain.

    ...
```

In bcc, we support kprobe function signatures in the bpf program.
The rewriter will automatically generate proper assignment to
these parameters. With the above function signature change, the
original method does not work any more.

This patch enhanced rewriter to generate two version codes guarded
with CONFIG_ARCH_HAS_SYSCALL_WRAPPER. But we need to identify
whether a function will be attached to syscall entry function
or not during prog load time at which time the program has not
attached to any event.

The prefix `kprobe__` is used for kprobe autoload, we can use
`kprobe____x64_sys_` as the prefix to identify x64 syscall entry
functions. To support other architecture or not-autoloading program,
the prefix `syscall__` is introduced to signal it is a syscall
entry function.

trace.py and other tools which uses kprobe syscall entry functions
are also modified with the new interface so that they can
work properly with 4.17.

Signed-off-by: Yonghong Song <yhs@fb.com>

Several python tools allow their eBPF code to be printed to
stdout for debugging.  There are other projects that would
like to share these program definitions however, instead of
duplicating code.  We previously agreed on an --ebpf option
and we now continue adding it to more tools.

Signed-off-by: Nathan Scott <nathans@redhat.com>

The $task refers to the current task. In my particular case, I need
to trace the number of users for file system associated with the current
task. With the probe alias $task, trace.py can easily trace this event

  ......
  trace.py -I 'linux/fs_struct.h' 'mntns_install "users = %d", $task->fs->users'
  PID     TID     COMM            FUNC             -
  176566  176566  python2.7       mntns_install    users = 2
  176566  176566  python2.7       mntns_install    users = 2
  ......

With $task probe alias, kernel tast_struct fields can be used in trace.py
filter or output easily even if they cannot be accessed through input
parameters.

Signed-off-by: Yonghong Song <yhs@fb.com>

Fix issue #1478
Two tools, trace.py and argdist.py, and their corresponding
example files were changed.
Fixed a minor typo in one of trace.py error messages.

Signed-off-by: Yonghong Song <yhs@fb.com>

`trace.py` parses a probe using the colon as a separator.  As a result, it
fails to create a uprobe for binary/library with a filename containing colons.

This diff fixes that issue with `trace.py`.  It requires a kernel with
https://lkml.org/lkml/2017/1/13/585 merged to work properly, otherwise
`trace.py` still fails for create uprobes.

This fixes the bcc module and all the affected tools for issues related to string handling in Python 3. Specifically, when passing Python strings to C libraries they are encoded as ASCII, and when constructing Python strings from C strings, they are decoded first.

* Travis CI build to check compliance with PEP8

* argdist: linter cleanup

* dbslower: linter cleanup

* dbstat: linter cleanup

* memleak: linter cleanup

* syscount: linter cleanup

* tplist: linter cleanup

* trace: linter cleanup

* ucalls: linter cleanup

* uflow: linter cleanup

* ugc: linter cleanup

* uobjnew: linter cleanup

* ustat: linter cleanup

As discussed in #966, this PR makes the size of the ring buffer used to send
data to userspace configurable. It changes the Python, Lua and C++ APIs to
expose this knob.

It also defaults the buffer size to a larger value (64 pages per CPU, an 8x
increase) for several tools which produce a lot of output, as well as making it
configurable in `trace` via a `-b` flag.

Adds support for resolving symbols using external debuginfo files,
which can be retrieved from two locations. First, check the build-id
of the desired binary and look in /usr/lib/debug/.build-id according
to the build-id structure. Second, check the debuglink section of
the desired binary and look in /usr/lib/debug or in the binary's
current directory. These are the rules applied by GDB as well, but
GDB lets the user reconfigure the debug directory path from
/usr/lib/debug to something else; we do not support this.

These changes are based on the following description of how GDB
resolves external debuginfo:

https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html

trace and argdist currently only work correctly for USDT arguments
whose size is exactly 8 bytes. Smaller types, such as chars, shorts,
ints (signed or unsigned) are not treated correctly. The reason is
that the produced program would invoke the `bpf_usdt_readarg` helper
with the address of a u64 local variable, and then cast that variable
to the user-specified type derived from the format string. However,
the `bpf_usdt_readarg` rewriting then passes `sizeof(u64)` to the
generated `bpf_..._readarg` macro, which then fails to read anything
because the provided size doesn't match the argument size it knows
about.

The fix is fairly easy: instead of declaring a u64 unconditionally
and reading into that variable with `bpf_usdt_readarg`, declare a
variable that has the correct type according to what we know about
the USDT probe.