Changes from version 2.4: Added three new commands: o trace-cmd stream This starts ftrace tracing and reads the binary files and processes it to produce text output without the need to save to disk (or network). o trace-cmd profile This uses the stream infrastructure from above, but also keeps track of the data and analyzes it to produce various useful statistics. By default it enables function graph tracing with a depth of 1, to only show where user tasks entered the kernel. It also enables sched switch and sched wakeup events to see how long it took to wake up a task. It also enables stack tracing to display where the latencies are happening. There's lots of options to change the default settings to show more about how the kernel is working. o trace-cmd stat Shows what may be enabled in ftrace. There's times when events are enabled or kprobes are added, or even when filters are used, and it may be difficult to see what state ftrace is in. Using trace-cmd stat will show you the state that ftrace is in. It will display if a tracer is enabled and much more. New options: In trace-cmd record Added --profile, that enables the default profiling events (and function graph tracing with depth of 1). These are the same events that are enabled with the trace-cmd profile command. In trace-cmd report Added --profile option that will read the data of the trace.dat file and report it the same as the trace-cmd profile command would. Some improvements: o trace-cmd record will reset everything it touches after it is done recording. Before it use to leave some things still active, like options and such. Now, if you enable options for recording, it will reset them when finished, as well as triggers and filters. trace-cmd start, or trace-cmd record with the -k option will leave the changes as is when done. o trace-cmd reset now resets triggers and filters.