Skip to content
Snippets Groups Projects
  1. Dec 22, 2020
  2. Aug 19, 2020
    • Andrei Ziureaev's avatar
      dt-bindings: Use json for processed-schema* · b8a49399
      Andrei Ziureaev authored
      
      Change the format of processed-schema* from yaml to json to speed up
      validation. With json output, using xargs and appending the output won't
      work since json has explicit list begin and end characters. Instead,
      we pass the schema files as a list in a temp file.
      
      The parsing time for the processed schema goes down from ~2sec to 70ms.
      Also, 'make dtbs_check' becomes 33% faster.
      
      Some error messages are affected by this change. For example, "True was
      expected" becomes "... is not of type 'boolean'". The order of messages
      is also changed.
      
      Signed-off-by: default avatarAndrei Ziureaev <andrei.ziureaev@arm.com>
      Signed-off-by: default avatarRob Herring <robh@kernel.org>
      b8a49399
  3. Aug 09, 2020
    • Masahiro Yamada's avatar
      kbuild: introduce hostprogs-always-y and userprogs-always-y · faabed29
      Masahiro Yamada authored
      
      To build host programs, you need to add the program names to 'hostprogs'
      to use the necessary build rule, but it is not enough to build them
      because there is no dependency.
      
      There are two types of host programs: built as the prerequisite of
      another (e.g. gen_crc32table in lib/Makefile), or always built when
      Kbuild visits the Makefile (e.g. genksyms in scripts/genksyms/Makefile).
      
      The latter is typical in Makefiles under scripts/, which contains host
      programs globally used during the kernel build. To build them, you need
      to add them to both 'hostprogs' and 'always-y'.
      
      This commit adds hostprogs-always-y as a shorthand.
      
      The same applies to user programs. net/bpfilter/Makefile builds
      bpfilter_umh on demand, hence always-y is unneeded. In contrast,
      programs under samples/ are added to both 'userprogs' and 'always-y'
      so they are always built when Kbuild visits the Makefiles.
      
      userprogs-always-y works as a shorthand.
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Acked-by: default avatarMiguel Ojeda <miguel.ojeda.sandonis@gmail.com>
      faabed29
    • Masahiro Yamada's avatar
      kbuild: introduce ccflags-remove-y and asflags-remove-y · 15d5761a
      Masahiro Yamada authored
      
      CFLAGS_REMOVE_<file>.o filters out flags when compiling a particular
      object, but there is no convenient way to do that for every object in
      a directory.
      
      Add ccflags-remove-y and asflags-remove-y to make it easily.
      
      Use ccflags-remove-y to clean up some Makefiles.
      
      The add/remove order works as follows:
      
       [1] KBUILD_CFLAGS specifies compiler flags used globally
      
       [2] ccflags-y adds compiler flags for all objects in the
           current Makefile
      
       [3] ccflags-remove-y removes compiler flags for all objects in the
           current Makefile (New feature)
      
       [4] CFLAGS_<file> adds compiler flags per file.
      
       [5] CFLAGS_REMOVE_<file> removes compiler flags per file.
      
      Having [3] before [4] allows us to remove flags from most (but not all)
      objects in the current Makefile.
      
      For example, kernel/trace/Makefile removes $(CC_FLAGS_FTRACE)
      from all objects in the directory, then adds it back to
      trace_selftest_dynamic.o and CFLAGS_trace_kprobe_selftest.o
      
      The same applies to lib/livepatch/Makefile.
      
      Please note ccflags-remove-y has no effect to the sub-directories.
      In contrast, the previous notation got rid of compiler flags also from
      all the sub-directories.
      
      The following are not affected because they have no sub-directories:
      
        arch/arm/boot/compressed/
        arch/powerpc/xmon/
        arch/sh/
        kernel/trace/
      
      However, lib/ has several sub-directories.
      
      To keep the behavior, I added ccflags-remove-y to all Makefiles
      in subdirectories of lib/, except the following:
      
        lib/vdso/Makefile        - Kbuild does not descend into this Makefile
        lib/raid/test/Makefile   - This is not used for the kernel build
      
      I think commit 2464a609 ("ftrace: do not trace library functions")
      excluded too much. In the next commit, I will remove ccflags-remove-y
      from the sub-directories of lib/.
      
      Suggested-by: default avatarSami Tolvanen <samitolvanen@google.com>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Acked-by: default avatarSteven Rostedt (VMware) <rostedt@goodmis.org>
      Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc)
      Acked-by: Brendan Higgins <brendanhiggins@google.com> (KUnit)
      Tested-by: default avatarAnders Roxell <anders.roxell@linaro.org>
      15d5761a
  4. Jul 31, 2020
  5. Jul 07, 2020
  6. Jun 30, 2020
    • Masahiro Yamada's avatar
      dt-bindings: copy process-schema-examples.yaml to process-schema.yaml · dee9c0b5
      Masahiro Yamada authored
      
      There are two processed schema files:
      
       - processed-schema-examples.yaml
      
          Used for 'make dt_binding_check'. This is always a full schema.
      
       - processed-schema.yaml
      
          Used for 'make dtbs_check'. This may be a full schema, or a smaller
          subset if DT_SCHEMA_FILES is given by a user.
      
      If DT_SCHEMA_FILES is not specified, they are the same. You can copy
      the former to the latter instead of running dt-mk-schema twice. This
      saves the cpu time a lot when you do 'make dt_binding_check dtbs_check'
      because building the full schema takes a couple of seconds.
      
      If DT_SCHEMA_FILES is specified, processed-schema.yaml is generated
      based on the specified yaml files.
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Link: https://lore.kernel.org/r/20200625170434.635114-4-masahiroy@kernel.org
      
      
      Signed-off-by: default avatarRob Herring <robh@kernel.org>
      dee9c0b5
    • Rob Herring's avatar
      scripts/dtc: Update to upstream version v1.6.0-11-g9d7888cbf19c · 3eb619b2
      Rob Herring authored
      
      Sync with upstream dtc primarily to pickup the I2C bus check fixes. The
      interrupt_provider check is noisy, so turn it off for now.
      
      This adds the following commits from upstream:
      
      9d7888cbf19c dtc: Consider one-character strings as strings
      8259d59f59de checks: Improve i2c reg property checking
      fdabcf2980a4 checks: Remove warning for I2C_OWN_SLAVE_ADDRESS
      2478b1652c8d libfdt: add extern "C" for C++
      f68bfc2668b2 libfdt: trivial typo fix
      7be250b4d059 libfdt: Correct condition for reordering blocks
      81e0919a3e21 checks: Add interrupt provider test
      85e5d839847a Makefile: when building libfdt only, do not add unneeded deps
      b28464a550c5 Fix some potential unaligned accesses in dtc
      
      Signed-off-by: default avatarRob Herring <robh@kernel.org>
      3eb619b2
  7. Jun 11, 2020
  8. Jun 06, 2020
    • Denis Efremov's avatar
      kbuild: add variables for compression tools · 8dfb61dc
      Denis Efremov authored
      
      Allow user to use alternative implementations of compression tools,
      such as pigz, pbzip2, pxz. For example, multi-threaded tools to
      speed up the build:
      $ make GZIP=pigz BZIP2=pbzip2
      
      Variables _GZIP, _BZIP2, _LZOP are used internally because original env
      vars are reserved by the tools. The use of GZIP in gzip tool is obsolete
      since 2015. However, alternative implementations (e.g., pigz) still rely
      on it. BZIP2, BZIP, LZOP vars are not obsolescent.
      
      The credit goes to @grsecurity.
      
      As a sidenote, for multi-threaded lzma, xz compression one can use:
      $ export XZ_OPT="--threads=0"
      
      Signed-off-by: default avatarDenis Efremov <efremov@linux.com>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      8dfb61dc
  9. Jun 03, 2020
  10. Jun 01, 2020
  11. May 25, 2020
  12. May 17, 2020
  13. May 12, 2020
    • Masahiro Yamada's avatar
      kbuild: determine the output format of DTC by the target suffix · 78046fab
      Masahiro Yamada authored
      
      cmd_dtc takes the additional parameter $(2) to select the target
      format, dtb or yaml. This makes things complicated when it is used
      with cmd_and_fixdep and if_changed_rule. I actually stumbled on this.
      See commit 3d4b2238 ("kbuild: fix DT binding schema rule again to
      avoid needless rebuilds").
      
      Extract the suffix part of the target instead of passing the parameter.
      Fortunately, this works for both $(obj)/%.dtb and $(obj)/%.dt.yaml .
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      78046fab
    • Masahiro Yamada's avatar
      kbuild: use -MMD instead of -MD to exclude system headers from dependency · 30a77297
      Masahiro Yamada authored
      
      This omits system headers from the generated header dependency.
      
      System headers are not updated unless you upgrade the compiler. Nor do
      they contain CONFIG options, so fixdep does not need to parse them.
      
      Having said that, the effect of this optimization will be quite small
      because the kernel code generally does not include system headers
      except <stdarg.h>. Host programs include a lot of system headers,
      but there are not so many in the kernel tree.
      
      At first, keeping system headers in .*.cmd files might be useful to
      detect the compiler update, but there is no guarantee that <stdarg.h>
      is included from every file. So, I implemented a more reliable way in
      the previous commit.
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      30a77297
  14. Apr 23, 2020
  15. Mar 03, 2020
  16. Feb 26, 2020
  17. Feb 03, 2020
    • Masahiro Yamada's avatar
      kbuild: rename hostprogs-y/always to hostprogs/always-y · 5f2fb52f
      Masahiro Yamada authored
      
      In old days, the "host-progs" syntax was used for specifying host
      programs. It was renamed to the current "hostprogs-y" in 2004.
      
      It is typically useful in scripts/Makefile because it allows Kbuild to
      selectively compile host programs based on the kernel configuration.
      
      This commit renames like follows:
      
        always       ->  always-y
        hostprogs-y  ->  hostprogs
      
      So, scripts/Makefile will look like this:
      
        always-$(CONFIG_BUILD_BIN2C) += ...
        always-$(CONFIG_KALLSYMS)    += ...
            ...
        hostprogs := $(always-y) $(always-m)
      
      I think this makes more sense because a host program is always a host
      program, irrespective of the kernel configuration. We want to specify
      which ones to compile by CONFIG options, so always-y will be handier.
      
      The "always", "hostprogs-y", "hostprogs-m" will be kept for backward
      compatibility for a while.
      
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      5f2fb52f
  18. Jan 06, 2020
  19. Nov 16, 2019
  20. Nov 14, 2019
    • Masahiro Yamada's avatar
      kbuild: remove header compile test · fcbb8461
      Masahiro Yamada authored
      There are both positive and negative options about this feature.
      At first, I thought it was a good idea, but actually Linus stated a
      negative opinion (https://lkml.org/lkml/2019/9/29/227
      
      ). I admit it
      is ugly and annoying.
      
      The baseline I'd like to keep is the compile-test of uapi headers.
      (Otherwise, kernel developers have no way to ensure the correctness
      of the exported headers.)
      
      I will maintain a small build rule in usr/include/Makefile.
      Remove the other header test functionality.
      
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      fcbb8461
  21. Oct 01, 2019
  22. Sep 06, 2019
  23. Sep 04, 2019
    • Masahiro Yamada's avatar
      kbuild: change *FLAGS_<basetarget>.o to take the path relative to $(obj) · 54b8ae66
      Masahiro Yamada authored
      
      Kbuild provides per-file compiler flag addition/removal:
      
        CFLAGS_<basetarget>.o
        CFLAGS_REMOVE_<basetarget>.o
        AFLAGS_<basetarget>.o
        AFLAGS_REMOVE_<basetarget>.o
        CPPFLAGS_<basetarget>.lds
        HOSTCFLAGS_<basetarget>.o
        HOSTCXXFLAGS_<basetarget>.o
      
      The <basetarget> is the filename of the target with its directory and
      suffix stripped.
      
      This syntax comes into a trouble when two files with the same basename
      appear in one Makefile, for example:
      
        obj-y += foo.o
        obj-y += dir/foo.o
        CFLAGS_foo.o := <some-flags>
      
      Here, the <some-flags> applies to both foo.o and dir/foo.o
      
      The real world problem is:
      
        scripts/kconfig/util.c
        scripts/kconfig/lxdialog/util.c
      
      Both files are compiled into scripts/kconfig/mconf, but only the
      latter should be given with the ncurses flags.
      
      It is more sensible to use the relative path to the Makefile, like this:
      
        obj-y += foo.o
        CFLAGS_foo.o := <some-flags>
        obj-y += dir/foo.o
        CFLAGS_dir/foo.o := <other-flags>
      
      At first, I attempted to replace $(basetarget) with $*. The $* variable
      is replaced with the stem ('%') part in a pattern rule. This works with
      most of cases, but does not for explicit rules.
      
      For example, arch/ia64/lib/Makefile reuses rule_as_o_S in its own
      explicit rules, so $* will be empty, resulting in ignoring the per-file
      AFLAGS.
      
      I introduced a new variable, target-stem, which can be used also from
      explicit rules.
      
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      Acked-by: default avatarMarc Zyngier <maz@kernel.org>
      54b8ae66
    • Masahiro Yamada's avatar
      kbuild: add $(BASH) to run scripts with bash-extension · 858805b3
      Masahiro Yamada authored
      
      CONFIG_SHELL falls back to sh when bash is not installed on the system,
      but nobody is testing such a case since bash is usually installed.
      So, shell scripts invoked by CONFIG_SHELL are only tested with bash.
      
      It makes it difficult to test whether the hashbang #!/bin/sh is real.
      For example, #!/bin/sh in arch/powerpc/kernel/prom_init_check.sh is
      false. (I fixed it up)
      
      Besides, some shell scripts invoked by CONFIG_SHELL use bash-extension
      and #!/bin/bash is specified as the hashbang, while CONFIG_SHELL may
      not always be set to bash.
      
      Probably, the right thing to do is to introduce BASH, which is bash by
      default, and always set CONFIG_SHELL to sh. Replace $(CONFIG_SHELL)
      with $(BASH) for bash scripts.
      
      If somebody tries to add bash-extension to a #!/bin/sh script, it will
      be caught in testing because /bin/sh is a symlink to dash on some major
      distributions.
      
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      858805b3
  24. Aug 21, 2019
  25. Aug 14, 2019
  26. Aug 13, 2019
    • Masahiro Yamada's avatar
      kbuild: move flex and bison rules to Makefile.host · cf8dfd15
      Masahiro Yamada authored
      
      Flex and bison are used for kconfig, dtc, genksyms, all of which are
      host programs. I never imagine the kernel embeds a parser or a lexer.
      
      Move the flex and bison rules to scripts/Makefile.host. This file is
      included only when hostprogs-y etc. is present in the Makefile in the
      directory. So, parsing these rules are skipped in most of directories.
      
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      cf8dfd15
    • Masahiro Yamada's avatar
      kbuild: make bison create C file and header in a single pattern rule · 6ba7dc66
      Masahiro Yamada authored
      We generally expect bison to create not only a C file, but also a
      header, which will be included from the lexer.
      
      Currently, Kbuild generates them in separate rules. So, for instance,
      when building Kconfig, you will notice bison is invoked twice:
      
        HOSTCC  scripts/kconfig/conf.o
        HOSTCC  scripts/kconfig/confdata.o
        HOSTCC  scripts/kconfig/expr.o
        LEX     scripts/kconfig/lexer.lex.c
        YACC    scripts/kconfig/parser.tab.h
        HOSTCC  scripts/kconfig/lexer.lex.o
        YACC    scripts/kconfig/parser.tab.c
        HOSTCC  scripts/kconfig/parser.tab.o
        HOSTCC  scripts/kconfig/preprocess.o
        HOSTCC  scripts/kconfig/symbol.o
        HOSTLD  scripts/kconfig/conf
      
      Make handles such cases nicely in pattern rules [1]. Merge the two
      rules so that one invokcation of bison can generate both of them.
      
        HOSTCC  scripts/kconfig/conf.o
        HOSTCC  scripts/kconfig/confdata.o
        HOSTCC  scripts/kconfig/expr.o
        LEX     scripts/kconfig/lexer.lex.c
        YACC    scripts/kconfig/parser.tab.[ch]
        HOSTCC  scripts/kconfig/lexer.lex.o
        HOSTCC  scripts/kconfig/parser.tab.o
        HOSTCC  scripts/kconfig/preprocess.o
        HOSTCC  scripts/kconfig/symbol.o
        HOSTLD  scripts/kconfig/conf
      
      [1] Pattern rule
      
      GNU Make manual says:
      "Pattern rules may have more than one target. Unlike normal rules,
      this does not act as many different rules with the same prerequisites
      and recipe. If a pattern rule has multiple targets, make knows that
      the rule's recipe is responsible for making all of the targets. The
      recipe is executed only once to make all the targets. When searching
      for a pattern rule to match a target, the target patterns of a rule
      other than the one that matches the target in need of a rule are
      incidental: make worries only about giving a recipe and prerequisites
      to the file presently in question. However, when this file's recipe is
      run, the other targets are marked as having been updated themselves."
      
      https://www.gnu.org/software/make/manual/html_node/Pattern-Intro.html
      
      
      
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      6ba7dc66
  27. Jul 27, 2019
  28. Jul 17, 2019
  29. Jul 10, 2019
  30. Jul 09, 2019
    • Masahiro Yamada's avatar
      kbuild: support header-test-pattern-y · 1e21cbfa
      Masahiro Yamada authored
      
      In my view, most of headers can be self-contained. So, it would be
      tedious to add every header to header-test-y explicitly. We usually
      end up with "all headers with some exceptions".
      
      There are two types in exceptions:
      
      [1] headers that are never compiled as standalone units
      
        For examples, include/linux/compiler-gcc.h is not intended for
        direct inclusion. We should always exclude such ones.
      
      [2] headers that are conditionally compiled as standalone units
      
        Some headers can be compiled only for particular architectures.
        For example, include/linux/arm-cci.h can be compiled only for
        arm/arm64 because it requires <asm/arm-cci.h> to exist.
        Clang can compile include/soc/nps/mtm.h only for arc because
        it contains an arch-specific register in inline assembler.
      
      So, you can write Makefile like this:
      
        header-test-                += linux/compiler-gcc.h
        header-test-$(CONFIG_ARM)   += linux/arm-cci.h
        header-test-$(CONFIG_ARM64) += linux/arm-cci.h
        header-test-$(CONFIG_ARC)   += soc/nps/mtm.h
      
      The new syntax header-test-pattern-y will be useful to specify
      "the rest".
      
      The typical usage is like this:
      
        header-test-pattern-y += */*.h
      
      This will add all the headers in sub-directories to the test coverage,
      excluding $(header-test-). In this regards, header-test-pattern-y
      behaves like a weaker variant of header-test-y.
      
      Caveat:
      The patterns in header-test-pattern-y are prefixed with $(srctree)/$(src)/
      but not $(objtree)/$(obj)/. Stale generated headers are often left over
      when you traverse the git history without cleaning. Wildcard patterns for
      $(objtree) may match to stale headers, which could fail to compile.
      One pitfall is $(srctree)/$(src)/ and $(objtree)/$(obj)/ point to the
      same directory for in-tree building. So, header-test-pattern-y should
      be used with care since it can potentially match to stale headers.
      
      Caveat2:
      You could use wildcard for header-test-. For example,
      
        header-test- += asm-generic/%
      
      ... will exclude headers in asm-generic directory. Unfortunately, the
      wildcard character is '%' instead of '*' here because this is evaluated
      by $(filter-out ...) whereas header-test-pattern-y is evaluated by
      $(wildcard ...). This is a kludge, but seems useful in some places...
      
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      Tested-by: default avatarJani Nikula <jani.nikula@intel.com>
      1e21cbfa
Loading