On Sun, Jun 14, 2020 at 11:26 PM kernel test robot <lkp(a)intel.com&gt; wrote: > > tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master > head: 96144c58abe7ff767e754b5b80995f7b8846d49b > commit: 39d010504e6b4485d7ceee167743620dd33f4417 net_sched: sch_fq: add horizon attribute > date: 6 weeks ago > :::::: branch date: 3 hours ago > :::::: commit date: 6 weeks ago > config: arm-randconfig-r006-20200614 (attached as .config) > compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project c669a1ed6386d57a75a602b53266466dae1e1d84) > reproduce (this is a W=1 build): > wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross > chmod +x ~/bin/make.cross > # install arm cross compiling tool for clang build > # apt-get install binutils-arm-linux-gnueabi > git checkout 39d010504e6b4485d7ceee167743620dd33f4417 > # save the attached .config to linux build tree > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm > > If you fix the issue, kindly add following tag as appropriate > Reported-by: kernel test robot <lkp(a)intel.com&gt; > > All warnings (new ones prefixed by >>, old ones prefixed by <<): > > >> net/sched/sch_fq.c:966:12: warning: stack frame size of 1400 bytes in function 'fq_dump' [-Wframe-larger-than=] > static int fq_dump(struct Qdisc *sch, struct sk_buff *skb) > ^ This looks like a bug in CLANG on ARM, there is no way fq_dump() could consume so much stack space.

> 1 warning generated. > > # https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit... > git remote add linus https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > git remote update linus > git checkout 39d010504e6b4485d7ceee167743620dd33f4417 > vim +/fq_dump +966 net/sched/sch_fq.c > > afe4fd062416b1 Eric Dumazet 2013-08-29 965 > afe4fd062416b1 Eric Dumazet 2013-08-29 @966 static int fq_dump(struct Qdisc *sch, struct sk_buff *skb) > afe4fd062416b1 Eric Dumazet 2013-08-29 967 { > afe4fd062416b1 Eric Dumazet 2013-08-29 968 struct fq_sched_data *q = qdisc_priv(sch); > 48872c11b77271 Eric Dumazet 2018-11-11 969 u64 ce_threshold = q->ce_threshold; > 39d010504e6b44 Eric Dumazet 2020-05-01 970 u64 horizon = q->horizon; > afe4fd062416b1 Eric Dumazet 2013-08-29 971 struct nlattr *opts; > afe4fd062416b1 Eric Dumazet 2013-08-29 972 > ae0be8de9a53cd Michal Kubecek 2019-04-26 973 opts = nla_nest_start_noflag(skb, TCA_OPTIONS); > afe4fd062416b1 Eric Dumazet 2013-08-29 974 if (opts == NULL) > afe4fd062416b1 Eric Dumazet 2013-08-29 975 goto nla_put_failure; > afe4fd062416b1 Eric Dumazet 2013-08-29 976 > 65c5189a2b57b9 Eric Dumazet 2013-11-15 977 /* TCA_FQ_FLOW_DEFAULT_RATE is not used anymore */ > 65c5189a2b57b9 Eric Dumazet 2013-11-15 978 > 48872c11b77271 Eric Dumazet 2018-11-11 979 do_div(ce_threshold, NSEC_PER_USEC); > 39d010504e6b44 Eric Dumazet 2020-05-01 980 do_div(horizon, NSEC_PER_USEC); > 48872c11b77271 Eric Dumazet 2018-11-11 981 > afe4fd062416b1 Eric Dumazet 2013-08-29 982 if (nla_put_u32(skb, TCA_FQ_PLIMIT, sch->limit) || > afe4fd062416b1 Eric Dumazet 2013-08-29 983 nla_put_u32(skb, TCA_FQ_FLOW_PLIMIT, q->flow_plimit) || > afe4fd062416b1 Eric Dumazet 2013-08-29 984 nla_put_u32(skb, TCA_FQ_QUANTUM, q->quantum) || > afe4fd062416b1 Eric Dumazet 2013-08-29 985 nla_put_u32(skb, TCA_FQ_INITIAL_QUANTUM, q->initial_quantum) || > afe4fd062416b1 Eric Dumazet 2013-08-29 986 nla_put_u32(skb, TCA_FQ_RATE_ENABLE, q->rate_enable) || > 76a9ebe811fb3d Eric Dumazet 2018-10-15 987 nla_put_u32(skb, TCA_FQ_FLOW_MAX_RATE, > 76a9ebe811fb3d Eric Dumazet 2018-10-15 988 min_t(unsigned long, q->flow_max_rate, ~0U)) || > f52ed89971adbe Eric Dumazet 2013-11-15 989 nla_put_u32(skb, TCA_FQ_FLOW_REFILL_DELAY, > f52ed89971adbe Eric Dumazet 2013-11-15 990 jiffies_to_usecs(q->flow_refill_delay)) || > 06eb395fa9856b Eric Dumazet 2015-02-04 991 nla_put_u32(skb, TCA_FQ_ORPHAN_MASK, q->orphan_mask) || > 77879147a3481b Eric Dumazet 2016-09-19 992 nla_put_u32(skb, TCA_FQ_LOW_RATE_THRESHOLD, > 77879147a3481b Eric Dumazet 2016-09-19 993 q->low_rate_threshold) || > 48872c11b77271 Eric Dumazet 2018-11-11 994 nla_put_u32(skb, TCA_FQ_CE_THRESHOLD, (u32)ce_threshold) || > 583396f4ca4d6e Eric Dumazet 2020-03-16 995 nla_put_u32(skb, TCA_FQ_BUCKETS_LOG, q->fq_trees_log) || > 39d010504e6b44 Eric Dumazet 2020-05-01 996 nla_put_u32(skb, TCA_FQ_TIMER_SLACK, q->timer_slack) || > 39d010504e6b44 Eric Dumazet 2020-05-01 997 nla_put_u32(skb, TCA_FQ_HORIZON, (u32)horizon) || > 39d010504e6b44 Eric Dumazet 2020-05-01 998 nla_put_u8(skb, TCA_FQ_HORIZON_DROP, q->horizon_drop)) > afe4fd062416b1 Eric Dumazet 2013-08-29 999 goto nla_put_failure; > afe4fd062416b1 Eric Dumazet 2013-08-29 1000 > d59b7d8059ddc4 Yang Yingliang 2014-03-12 1001 return nla_nest_end(skb, opts); > afe4fd062416b1 Eric Dumazet 2013-08-29 1002 > afe4fd062416b1 Eric Dumazet 2013-08-29 1003 nla_put_failure: > afe4fd062416b1 Eric Dumazet 2013-08-29 1004 return -1; > afe4fd062416b1 Eric Dumazet 2013-08-29 1005 } > afe4fd062416b1 Eric Dumazet 2013-08-29 1006 > > :::::: The code at line 966 was first introduced by commit > :::::: afe4fd062416b158a8a8538b23adc1930a9b88dc pkt_sched: fq: Fair Queue packet scheduler > > :::::: TO: Eric Dumazet <edumazet(a)google.com&gt; > :::::: CC: David S. Miller <davem(a)davemloft.net&gt; > > --- > 0-DAY CI Kernel Test Service, Intel Corporation > https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org > _______________________________________________ > kbuild mailing list -- kbuild(a)lists.01.org > To unsubscribe send an email to kbuild-leave(a)lists.01.org -- You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe(a)googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CANn89i%2Bs%3DoFTq6KF....

On Mon, Jun 15, 2020 at 10:44 AM Nick Desaulniers <ndesaulniers(a)google.com&gt; wrote: > > On Mon, Jun 15, 2020 at 9:17 AM 'Eric Dumazet' via Clang Built Linux > <clang-built-linux(a)googlegroups.com&gt; wrote: > > > > On Sun, Jun 14, 2020 at 11:26 PM kernel test robot <lkp(a)intel.com&gt; wrote: > > > > > > tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master > > > head: 96144c58abe7ff767e754b5b80995f7b8846d49b > > > commit: 39d010504e6b4485d7ceee167743620dd33f4417 net_sched: sch_fq: add horizon attribute > > > date: 6 weeks ago > > > :::::: branch date: 3 hours ago > > > :::::: commit date: 6 weeks ago > > > config: arm-randconfig-r006-20200614 (attached as .config) > > > compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project c669a1ed6386d57a75a602b53266466dae1e1d84) > > > reproduce (this is a W=1 build): > > > wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross > > > chmod +x ~/bin/make.cross > > > # install arm cross compiling tool for clang build > > > # apt-get install binutils-arm-linux-gnueabi > > > git checkout 39d010504e6b4485d7ceee167743620dd33f4417 > > > # save the attached .config to linux build tree > > > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm > > > > > > If you fix the issue, kindly add following tag as appropriate > > > Reported-by: kernel test robot <lkp(a)intel.com&gt; > > > > > > All warnings (new ones prefixed by >>, old ones prefixed by <<): > > > > > > >> net/sched/sch_fq.c:966:12: warning: stack frame size of 1400 bytes in function 'fq_dump' [-Wframe-larger-than=] > > > static int fq_dump(struct Qdisc *sch, struct sk_buff *skb) > > > ^ > > > > > > This looks like a bug in CLANG on ARM, there is no way fq_dump() could > > consume so much stack space. > > You can use > https://github.com/ClangBuiltLinux/frame-larger-than > to help debug these. You might be surprised who's doing larger stack > allocations than expected. What is wrong with scripts/checkstack.pl ?

On Mon, Jun 15, 2020 at 10:59 AM 'Eric Dumazet' via Clang Built Linux <clang-built-linux(a)googlegroups.com&gt; wrote: > > On Mon, Jun 15, 2020 at 10:54 AM Eric Dumazet <edumazet(a)google.com&gt; wrote: > > > > On Mon, Jun 15, 2020 at 10:44 AM Nick Desaulniers > > <ndesaulniers(a)google.com&gt; wrote: > > > > > > On Mon, Jun 15, 2020 at 9:17 AM 'Eric Dumazet' via Clang Built Linux > > > <clang-built-linux(a)googlegroups.com&gt; wrote: > > > > > > > > On Sun, Jun 14, 2020 at 11:26 PM kernel test robot <lkp(a)intel.com&gt; wrote: > > > > > > > > > > tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master > > > > > head: 96144c58abe7ff767e754b5b80995f7b8846d49b > > > > > commit: 39d010504e6b4485d7ceee167743620dd33f4417 net_sched: sch_fq: add horizon attribute > > > > > date: 6 weeks ago > > > > > :::::: branch date: 3 hours ago > > > > > :::::: commit date: 6 weeks ago > > > > > config: arm-randconfig-r006-20200614 (attached as .config) > > > > > compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project c669a1ed6386d57a75a602b53266466dae1e1d84) > > > > > reproduce (this is a W=1 build): > > > > > wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross > > > > > chmod +x ~/bin/make.cross > > > > > # install arm cross compiling tool for clang build > > > > > # apt-get install binutils-arm-linux-gnueabi > > > > > git checkout 39d010504e6b4485d7ceee167743620dd33f4417 > > > > > # save the attached .config to linux build tree > > > > > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm > > > > > > > > > > If you fix the issue, kindly add following tag as appropriate > > > > > Reported-by: kernel test robot <lkp(a)intel.com&gt; > > > > > > > > > > All warnings (new ones prefixed by >>, old ones prefixed by <<): > > > > > > > > > > >> net/sched/sch_fq.c:966:12: warning: stack frame size of 1400 bytes in function 'fq_dump' [-Wframe-larger-than=] > > > > > static int fq_dump(struct Qdisc *sch, struct sk_buff *skb) > > > > > ^ > > > > > > > > > > > > This looks like a bug in CLANG on ARM, there is no way fq_dump() could > > > > consume so much stack space. > > > > > > You can use > > > https://github.com/ClangBuiltLinux/frame-larger-than > > > to help debug these. You might be surprised who's doing larger stack > > > allocations than expected. > > > > What is wrong with scripts/checkstack.pl ? > > After using CLANG on x86 I get : > > $ objdump -d net/sched/sch_fq.o | scripts/checkstack.pl > 0x00001136 fq_change [sch_fq.o]: 192 > 0x00000016 __direct_call_Qdisc_enqueue1 [sch_fq.o]: 112 > 0x000019e0 fq_dump_stats [sch_fq.o]: 112 > 0x00001b43 fq_dump_stats [sch_fq.o]: 112 > > So CLANG on x86 seems fine. > > I said : " this looks like a bug in CLANG on ARM" This is a randconfig, so some option is turning on something that's causing excessive stack usage with your patch on ARM. Building who knows what config with x86 and claiming it works doesn't really cut it. You looked at fq_dump and claimed it should be fine. Did you verify each called function that was inlined?

And the issue with checkstack is it tells you the depth, but not *what variables* and from which inlined function may be the cause.

On Mon, Jun 15, 2020 at 11:45 AM Eric Dumazet <edumazet(a)google.com&gt; wrote: > > On Mon, Jun 15, 2020 at 11:07 AM Nick Desaulniers > <ndesaulniers(a)google.com&gt; wrote: > > > > On Mon, Jun 15, 2020 at 10:59 AM 'Eric Dumazet' via Clang Built Linux > > <clang-built-linux(a)googlegroups.com&gt; wrote: > > > > > > On Mon, Jun 15, 2020 at 10:54 AM Eric Dumazet <edumazet(a)google.com&gt; wrote: > > > > > > > > On Mon, Jun 15, 2020 at 10:44 AM Nick Desaulniers > > > > <ndesaulniers(a)google.com&gt; wrote: > > > > > > > > > > On Mon, Jun 15, 2020 at 9:17 AM 'Eric Dumazet' via Clang Built Linux > > > > > <clang-built-linux(a)googlegroups.com&gt; wrote: > > > > > > > > > > > > On Sun, Jun 14, 2020 at 11:26 PM kernel test robot <lkp(a)intel.com&gt; wrote: > > > > > > > > > > > > > > tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master > > > > > > > head: 96144c58abe7ff767e754b5b80995f7b8846d49b > > > > > > > commit: 39d010504e6b4485d7ceee167743620dd33f4417 net_sched: sch_fq: add horizon attribute > > > > > > > date: 6 weeks ago > > > > > > > :::::: branch date: 3 hours ago > > > > > > > :::::: commit date: 6 weeks ago > > > > > > > config: arm-randconfig-r006-20200614 (attached as .config) > > > > > > > compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project c669a1ed6386d57a75a602b53266466dae1e1d84) > > > > > > > reproduce (this is a W=1 build): > > > > > > > wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross > > > > > > > chmod +x ~/bin/make.cross > > > > > > > # install arm cross compiling tool for clang build > > > > > > > # apt-get install binutils-arm-linux-gnueabi > > > > > > > git checkout 39d010504e6b4485d7ceee167743620dd33f4417 > > > > > > > # save the attached .config to linux build tree > > > > > > > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm > > > > > > > > > > > > > > If you fix the issue, kindly add following tag as appropriate > > > > > > > Reported-by: kernel test robot <lkp(a)intel.com&gt; > > > > > > > > > > > > > > All warnings (new ones prefixed by >>, old ones prefixed by <<): > > > > > > > > > > > > > > >> net/sched/sch_fq.c:966:12: warning: stack frame size of 1400 bytes in function 'fq_dump' [-Wframe-larger-than=] > > > > > > > static int fq_dump(struct Qdisc *sch, struct sk_buff *skb) > > > > > > > ^ > > > > > > > > > > > > > > > > > > This looks like a bug in CLANG on ARM, there is no way fq_dump() could > > > > > > consume so much stack space. > > > > > > > > > > You can use > > > > > https://github.com/ClangBuiltLinux/frame-larger-than > > > > > to help debug these. You might be surprised who's doing larger stack > > > > > allocations than expected. > > > > > > > > What is wrong with scripts/checkstack.pl ? > > > > > > After using CLANG on x86 I get : > > > > > > $ objdump -d net/sched/sch_fq.o | scripts/checkstack.pl > > > 0x00001136 fq_change [sch_fq.o]: 192 > > > 0x00000016 __direct_call_Qdisc_enqueue1 [sch_fq.o]: 112 > > > 0x000019e0 fq_dump_stats [sch_fq.o]: 112 > > > 0x00001b43 fq_dump_stats [sch_fq.o]: 112 > > > > > > So CLANG on x86 seems fine. > > > > > > I said : " this looks like a bug in CLANG on ARM" > > > > This is a randconfig, so some option is turning on something that's > > causing excessive stack usage with your patch on ARM. Building who > > knows what config with x86 and claiming it works doesn't really cut > > it. You looked at fq_dump and claimed it should be fine. Did you > > verify each called function that was inlined? > > Nick, my patch added exactly _one_ u64 variable in the stack. > That's a whooping 8 bytes, right ? What if I add a single byte of stack usage, and that's enough to trip this warning? Not my problem? > > We should be able to use as many nla_put_u32() calls in a function, > without changing stack sizes. That's good to know, see below. (That not be the case here). > > If CLANG can not do that, then this is a CLANG problem, I have no > intention of fixing it, > I hope this is very clear ;) At least you tested it with Clang, so I appreciate you taking the time to do that. Here's how we might go about debugging this further: $ git checkout 39d010504e6b4485d7ceee167743620dd33f4417 $ wget https://lore.kernel.org/lkml/20200615062520.GK12456@shao2-debian/2-a.bin -O .config.gz $ gunzip .config.gz $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang -j71 net/sched/sch_fq.o ... net/sched/sch_fq.c:966:12: warning: stack frame size of 1400 bytes in function 'fq_dump' [-Wframe-larger-than=] $ python3 /path/to/frame_larger_than.py net/sched/sch_fq.o fq_dump fq_dump: 8 u64 horizon 8 u64 ce_threshold 4 struct fq_sched_data* q 4 struct nlattr* opts qdisc_priv: nla_nest_start_noflag: 4 struct nlattr* start 4 struct nlattr* start skb_tail_pointer: 4 uint32_t __base 4 uint32_t __rem is_power_of_2: 4 uint32_t __n_lo 4 uint32_t __res_lo 8 uint64_t ___n 8 uint64_t ___m 4 uint32_t ___bias 8 uint64_t ___res 8 uint64_t ___t 8 uint64_t ___x 4 uint32_t ___p __arch_xprod_64: 4 unsigned int tmp 8 long long unsigned int res 4 unsigned int tmp 8 long long unsigned int res 4 uint32_t __base 4 uint32_t __rem is_power_of_2: 4 uint32_t __n_lo 4 uint32_t __res_lo 8 uint64_t ___n 8 uint64_t ___res 8 uint64_t ___t 8 uint64_t ___x 8 uint64_t ___m 4 uint32_t ___p 4 uint32_t ___bias __arch_xprod_64: 4 unsigned int tmp 8 long long unsigned int res 4 unsigned int tmp 8 long long unsigned int res nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u32: 4 u32 tmp 4 u32 tmp 4 long unsigned int __UNIQUE_ID___y256 4 long unsigned int __UNIQUE_ID___x255 nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u32: 4 u32 tmp 4 u32 tmp nla_put_u8: 1 u8 tmp 1 u8 tmp nla_nest_end: skb_tail_pointer: So is_power_of_2/__arch_xprod_64 looks excessively large, and the nla_put_u* routines seem to have 2 temporary variables? (I should update the tool to better denote the end of an inlined frame; I don't think is_power_of_2 is the source, it looks like do_div() calls with 64b non-integer constant expressions. Having those be macros vs noinline_for_stack might be of interest for the non-ICE case, cc Arnd). Let me see if I can extend the tool to print file+line numbers so we can see if those `tmp`s are one in the same or not. > > > > > > > And the issue with checkstack is it tells you the depth, but not *what > > variables* and from which inlined function may be the cause. > > > OK. Thank you. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1 warning generated. > > > > > > > > > > > > > > # https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit... > > > > > > > git remote add linus https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > > > > > > > git remote update linus > > > > > > > git checkout 39d010504e6b4485d7ceee167743620dd33f4417 > > > > > > > vim +/fq_dump +966 net/sched/sch_fq.c > > > > > > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 965 > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 @966 static int fq_dump(struct Qdisc *sch, struct sk_buff *skb) > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 967 { > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 968 struct fq_sched_data *q = qdisc_priv(sch); > > > > > > > 48872c11b77271 Eric Dumazet 2018-11-11 969 u64 ce_threshold = q->ce_threshold; > > > > > > > 39d010504e6b44 Eric Dumazet 2020-05-01 970 u64 horizon = q->horizon; > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 971 struct nlattr *opts; > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 972 > > > > > > > ae0be8de9a53cd Michal Kubecek 2019-04-26 973 opts = nla_nest_start_noflag(skb, TCA_OPTIONS); > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 974 if (opts == NULL) > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 975 goto nla_put_failure; > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 976 > > > > > > > 65c5189a2b57b9 Eric Dumazet 2013-11-15 977 /* TCA_FQ_FLOW_DEFAULT_RATE is not used anymore */ > > > > > > > 65c5189a2b57b9 Eric Dumazet 2013-11-15 978 > > > > > > > 48872c11b77271 Eric Dumazet 2018-11-11 979 do_div(ce_threshold, NSEC_PER_USEC); > > > > > > > 39d010504e6b44 Eric Dumazet 2020-05-01 980 do_div(horizon, NSEC_PER_USEC); > > > > > > > 48872c11b77271 Eric Dumazet 2018-11-11 981 > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 982 if (nla_put_u32(skb, TCA_FQ_PLIMIT, sch->limit) || > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 983 nla_put_u32(skb, TCA_FQ_FLOW_PLIMIT, q->flow_plimit) || > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 984 nla_put_u32(skb, TCA_FQ_QUANTUM, q->quantum) || > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 985 nla_put_u32(skb, TCA_FQ_INITIAL_QUANTUM, q->initial_quantum) || > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 986 nla_put_u32(skb, TCA_FQ_RATE_ENABLE, q->rate_enable) || > > > > > > > 76a9ebe811fb3d Eric Dumazet 2018-10-15 987 nla_put_u32(skb, TCA_FQ_FLOW_MAX_RATE, > > > > > > > 76a9ebe811fb3d Eric Dumazet 2018-10-15 988 min_t(unsigned long, q->flow_max_rate, ~0U)) || > > > > > > > f52ed89971adbe Eric Dumazet 2013-11-15 989 nla_put_u32(skb, TCA_FQ_FLOW_REFILL_DELAY, > > > > > > > f52ed89971adbe Eric Dumazet 2013-11-15 990 jiffies_to_usecs(q->flow_refill_delay)) || > > > > > > > 06eb395fa9856b Eric Dumazet 2015-02-04 991 nla_put_u32(skb, TCA_FQ_ORPHAN_MASK, q->orphan_mask) || > > > > > > > 77879147a3481b Eric Dumazet 2016-09-19 992 nla_put_u32(skb, TCA_FQ_LOW_RATE_THRESHOLD, > > > > > > > 77879147a3481b Eric Dumazet 2016-09-19 993 q->low_rate_threshold) || > > > > > > > 48872c11b77271 Eric Dumazet 2018-11-11 994 nla_put_u32(skb, TCA_FQ_CE_THRESHOLD, (u32)ce_threshold) || > > > > > > > 583396f4ca4d6e Eric Dumazet 2020-03-16 995 nla_put_u32(skb, TCA_FQ_BUCKETS_LOG, q->fq_trees_log) || > > > > > > > 39d010504e6b44 Eric Dumazet 2020-05-01 996 nla_put_u32(skb, TCA_FQ_TIMER_SLACK, q->timer_slack) || > > > > > > > 39d010504e6b44 Eric Dumazet 2020-05-01 997 nla_put_u32(skb, TCA_FQ_HORIZON, (u32)horizon) || > > > > > > > 39d010504e6b44 Eric Dumazet 2020-05-01 998 nla_put_u8(skb, TCA_FQ_HORIZON_DROP, q->horizon_drop)) > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 999 goto nla_put_failure; > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1000 > > > > > > > d59b7d8059ddc4 Yang Yingliang 2014-03-12 1001 return nla_nest_end(skb, opts); > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1002 > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1003 nla_put_failure: > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1004 return -1; > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1005 } > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1006 > > > > > > > > > > > > > > :::::: The code at line 966 was first introduced by commit > > > > > > > :::::: afe4fd062416b158a8a8538b23adc1930a9b88dc pkt_sched: fq: Fair Queue packet scheduler > > > > > > > > > > > > > > :::::: TO: Eric Dumazet <edumazet(a)google.com&gt; > > > > > > > :::::: CC: David S. Miller <davem(a)davemloft.net&gt; > > > > > > > > > > > > > > --- > > > > > > > 0-DAY CI Kernel Test Service, Intel Corporation > > > > > > > https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org > > > > > > > _______________________________________________ > > > > > > > kbuild mailing list -- kbuild(a)lists.01.org > > > > > > > To unsubscribe send an email to kbuild-leave(a)lists.01.org > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe(a)googlegroups.com. > > > > > > To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CANn89i%2Bs%3DoFTq6KF.... > > > > > > > > > > > > > > > > > > > > -- > > > > > Thanks, > > > > > ~Nick Desaulniers > > > > > > -- > > > You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. > > > To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe(a)googlegroups.com. > > > To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CANn89iLnmKaa30tPoyK_.... > > > > > > > > -- > > Thanks, > > ~Nick Desaulniers -- Thanks, ~Nick Desaulniers

On Mon, Jun 15, 2020 at 1:37 PM Nick Desaulniers <ndesaulniers(a)google.com&gt; wrote: > > On Mon, Jun 15, 2020 at 1:36 PM Nick Desaulniers > <ndesaulniers(a)google.com&gt; wrote: > > > > On Mon, Jun 15, 2020 at 11:45 AM Eric Dumazet <edumazet(a)google.com&gt; wrote: > > > > > > On Mon, Jun 15, 2020 at 11:07 AM Nick Desaulniers > > > <ndesaulniers(a)google.com&gt; wrote: > > > > > > > > On Mon, Jun 15, 2020 at 10:59 AM 'Eric Dumazet' via Clang Built Linux > > > > <clang-built-linux(a)googlegroups.com&gt; wrote: > > > > > > > > > > On Mon, Jun 15, 2020 at 10:54 AM Eric Dumazet <edumazet(a)google.com&gt; wrote: > > > > > > > > > > > > On Mon, Jun 15, 2020 at 10:44 AM Nick Desaulniers > > > > > > <ndesaulniers(a)google.com&gt; wrote: > > > > > > > > > > > > > > On Mon, Jun 15, 2020 at 9:17 AM 'Eric Dumazet' via Clang Built Linux > > > > > > > <clang-built-linux(a)googlegroups.com&gt; wrote: > > > > > > > > > > > > > > > > On Sun, Jun 14, 2020 at 11:26 PM kernel test robot <lkp(a)intel.com&gt; wrote: > > > > > > > > > > > > > > > > > > tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master > > > > > > > > > head: 96144c58abe7ff767e754b5b80995f7b8846d49b > > > > > > > > > commit: 39d010504e6b4485d7ceee167743620dd33f4417 net_sched: sch_fq: add horizon attribute > > > > > > > > > date: 6 weeks ago > > > > > > > > > :::::: branch date: 3 hours ago > > > > > > > > > :::::: commit date: 6 weeks ago > > > > > > > > > config: arm-randconfig-r006-20200614 (attached as .config) > > > > > > > > > compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project c669a1ed6386d57a75a602b53266466dae1e1d84) > > > > > > > > > reproduce (this is a W=1 build): > > > > > > > > > wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross > > > > > > > > > chmod +x ~/bin/make.cross > > > > > > > > > # install arm cross compiling tool for clang build > > > > > > > > > # apt-get install binutils-arm-linux-gnueabi > > > > > > > > > git checkout 39d010504e6b4485d7ceee167743620dd33f4417 > > > > > > > > > # save the attached .config to linux build tree > > > > > > > > > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm > > > > > > > > > > > > > > > > > > If you fix the issue, kindly add following tag as appropriate > > > > > > > > > Reported-by: kernel test robot <lkp(a)intel.com&gt; > > > > > > > > > > > > > > > > > > All warnings (new ones prefixed by >>, old ones prefixed by <<): > > > > > > > > > > > > > > > > > > >> net/sched/sch_fq.c:966:12: warning: stack frame size of 1400 bytes in function 'fq_dump' [-Wframe-larger-than=] > > > > > > > > > static int fq_dump(struct Qdisc *sch, struct sk_buff *skb) > > > > > > > > > ^ > > > > > > > > > > > > > > > > > > > > > > > > This looks like a bug in CLANG on ARM, there is no way fq_dump() could > > > > > > > > consume so much stack space. > > > > > > > > > > > > > > You can use > > > > > > > https://github.com/ClangBuiltLinux/frame-larger-than > > > > > > > to help debug these. You might be surprised who's doing larger stack > > > > > > > allocations than expected. > > > > > > > > > > > > What is wrong with scripts/checkstack.pl ? > > > > > > > > > > After using CLANG on x86 I get : > > > > > > > > > > $ objdump -d net/sched/sch_fq.o | scripts/checkstack.pl > > > > > 0x00001136 fq_change [sch_fq.o]: 192 > > > > > 0x00000016 __direct_call_Qdisc_enqueue1 [sch_fq.o]: 112 > > > > > 0x000019e0 fq_dump_stats [sch_fq.o]: 112 > > > > > 0x00001b43 fq_dump_stats [sch_fq.o]: 112 > > > > > > > > > > So CLANG on x86 seems fine. > > > > > > > > > > I said : " this looks like a bug in CLANG on ARM" > > > > > > > > This is a randconfig, so some option is turning on something that's > > > > causing excessive stack usage with your patch on ARM. Building who > > > > knows what config with x86 and claiming it works doesn't really cut > > > > it. You looked at fq_dump and claimed it should be fine. Did you > > > > verify each called function that was inlined? > > > > > > Nick, my patch added exactly _one_ u64 variable in the stack. > > > That's a whooping 8 bytes, right ? > > > > What if I add a single byte of stack usage, and that's enough to trip > > this warning? Not my problem? > > > > > > > > We should be able to use as many nla_put_u32() calls in a function, > > > without changing stack sizes. > > > > That's good to know, see below. (That not be the case here). > > > > > > > > If CLANG can not do that, then this is a CLANG problem, I have no > > > intention of fixing it, > > > I hope this is very clear ;) > > > > At least you tested it with Clang, so I appreciate you taking the time > > to do that. Here's how we might go about debugging this further: > > > > $ git checkout 39d010504e6b4485d7ceee167743620dd33f4417 > > $ wget https://lore.kernel.org/lkml/20200615062520.GK12456@shao2-debian/2-a.bin > > -O .config.gz > > $ gunzip .config.gz Also, it looks like CONFIG_UBSAN=y was set in this randconfig, so that may play a part in the excessive stack usage. We've definitely seen that with CONFIG_KASAN=y.

> > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang -j71 > > net/sched/sch_fq.o > > ... > > net/sched/sch_fq.c:966:12: warning: stack frame size of 1400 bytes in > > function 'fq_dump' [-Wframe-larger-than=] > > $ python3 /path/to/frame_larger_than.py net/sched/sch_fq.o fq_dump > > fq_dump: > > 8 u64 horizon > > 8 u64 ce_threshold > > 4 struct fq_sched_data* q > > 4 struct nlattr* opts > > qdisc_priv: > > nla_nest_start_noflag: > > 4 struct nlattr* start > > 4 struct nlattr* start > > skb_tail_pointer: > > 4 uint32_t __base > > 4 uint32_t __rem > > is_power_of_2: > > 4 uint32_t __n_lo > > 4 uint32_t __res_lo > > 8 uint64_t ___n > > 8 uint64_t ___m > > 4 uint32_t ___bias > > 8 uint64_t ___res > > 8 uint64_t ___t > > 8 uint64_t ___x > > 4 uint32_t ___p > > __arch_xprod_64: > > 4 unsigned int tmp > > 8 long long unsigned int res > > 4 unsigned int tmp > > 8 long long unsigned int res > > 4 uint32_t __base > > 4 uint32_t __rem > > is_power_of_2: > > 4 uint32_t __n_lo > > 4 uint32_t __res_lo > > 8 uint64_t ___n > > 8 uint64_t ___res > > 8 uint64_t ___t > > 8 uint64_t ___x > > 8 uint64_t ___m > > 4 uint32_t ___p > > 4 uint32_t ___bias > > __arch_xprod_64: > > 4 unsigned int tmp > > 8 long long unsigned int res > > 4 unsigned int tmp > > 8 long long unsigned int res > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > 4 long unsigned int __UNIQUE_ID___y256 > > 4 long unsigned int __UNIQUE_ID___x255 > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u32: > > 4 u32 tmp > > 4 u32 tmp > > nla_put_u8: > > 1 u8 tmp > > 1 u8 tmp > > nla_nest_end: > > skb_tail_pointer: > > > > So is_power_of_2/__arch_xprod_64 looks excessively large, and the > > nla_put_u* routines seem to have 2 temporary variables? > > > > (I should update the tool to better denote the end of an inlined > > frame; I don't think is_power_of_2 is the source, it looks like > > do_div() calls with 64b non-integer constant expressions. Having those > > be macros vs noinline_for_stack might be of interest for the non-ICE > > case, cc Arnd). > > > > Let me see if I can extend the tool to print file+line numbers so we > > can see if those `tmp`s are one in the same or not. > > > > > > > > > > > > > > > > > And the issue with checkstack is it tells you the depth, but not *what > > > > variables* and from which inlined function may be the cause. > > > > > > > > > OK. Thank you. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1 warning generated. > > > > > > > > > > > > > > > > > > # https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit... > > > > > > > > > git remote add linus https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > > > > > > > > > git remote update linus > > > > > > > > > git checkout 39d010504e6b4485d7ceee167743620dd33f4417 > > > > > > > > > vim +/fq_dump +966 net/sched/sch_fq.c > > > > > > > > > > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 965 > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 @966 static int fq_dump(struct Qdisc *sch, struct sk_buff *skb) > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 967 { > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 968 struct fq_sched_data *q = qdisc_priv(sch); > > > > > > > > > 48872c11b77271 Eric Dumazet 2018-11-11 969 u64 ce_threshold = q->ce_threshold; > > > > > > > > > 39d010504e6b44 Eric Dumazet 2020-05-01 970 u64 horizon = q->horizon; > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 971 struct nlattr *opts; > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 972 > > > > > > > > > ae0be8de9a53cd Michal Kubecek 2019-04-26 973 opts = nla_nest_start_noflag(skb, TCA_OPTIONS); > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 974 if (opts == NULL) > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 975 goto nla_put_failure; > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 976 > > > > > > > > > 65c5189a2b57b9 Eric Dumazet 2013-11-15 977 /* TCA_FQ_FLOW_DEFAULT_RATE is not used anymore */ > > > > > > > > > 65c5189a2b57b9 Eric Dumazet 2013-11-15 978 > > > > > > > > > 48872c11b77271 Eric Dumazet 2018-11-11 979 do_div(ce_threshold, NSEC_PER_USEC); > > > > > > > > > 39d010504e6b44 Eric Dumazet 2020-05-01 980 do_div(horizon, NSEC_PER_USEC); > > > > > > > > > 48872c11b77271 Eric Dumazet 2018-11-11 981 > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 982 if (nla_put_u32(skb, TCA_FQ_PLIMIT, sch->limit) || > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 983 nla_put_u32(skb, TCA_FQ_FLOW_PLIMIT, q->flow_plimit) || > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 984 nla_put_u32(skb, TCA_FQ_QUANTUM, q->quantum) || > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 985 nla_put_u32(skb, TCA_FQ_INITIAL_QUANTUM, q->initial_quantum) || > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 986 nla_put_u32(skb, TCA_FQ_RATE_ENABLE, q->rate_enable) || > > > > > > > > > 76a9ebe811fb3d Eric Dumazet 2018-10-15 987 nla_put_u32(skb, TCA_FQ_FLOW_MAX_RATE, > > > > > > > > > 76a9ebe811fb3d Eric Dumazet 2018-10-15 988 min_t(unsigned long, q->flow_max_rate, ~0U)) || > > > > > > > > > f52ed89971adbe Eric Dumazet 2013-11-15 989 nla_put_u32(skb, TCA_FQ_FLOW_REFILL_DELAY, > > > > > > > > > f52ed89971adbe Eric Dumazet 2013-11-15 990 jiffies_to_usecs(q->flow_refill_delay)) || > > > > > > > > > 06eb395fa9856b Eric Dumazet 2015-02-04 991 nla_put_u32(skb, TCA_FQ_ORPHAN_MASK, q->orphan_mask) || > > > > > > > > > 77879147a3481b Eric Dumazet 2016-09-19 992 nla_put_u32(skb, TCA_FQ_LOW_RATE_THRESHOLD, > > > > > > > > > 77879147a3481b Eric Dumazet 2016-09-19 993 q->low_rate_threshold) || > > > > > > > > > 48872c11b77271 Eric Dumazet 2018-11-11 994 nla_put_u32(skb, TCA_FQ_CE_THRESHOLD, (u32)ce_threshold) || > > > > > > > > > 583396f4ca4d6e Eric Dumazet 2020-03-16 995 nla_put_u32(skb, TCA_FQ_BUCKETS_LOG, q->fq_trees_log) || > > > > > > > > > 39d010504e6b44 Eric Dumazet 2020-05-01 996 nla_put_u32(skb, TCA_FQ_TIMER_SLACK, q->timer_slack) || > > > > > > > > > 39d010504e6b44 Eric Dumazet 2020-05-01 997 nla_put_u32(skb, TCA_FQ_HORIZON, (u32)horizon) || > > > > > > > > > 39d010504e6b44 Eric Dumazet 2020-05-01 998 nla_put_u8(skb, TCA_FQ_HORIZON_DROP, q->horizon_drop)) > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 999 goto nla_put_failure; > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1000 > > > > > > > > > d59b7d8059ddc4 Yang Yingliang 2014-03-12 1001 return nla_nest_end(skb, opts); > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1002 > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1003 nla_put_failure: > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1004 return -1; > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1005 } > > > > > > > > > afe4fd062416b1 Eric Dumazet 2013-08-29 1006 > > > > > > > > > > > > > > > > > > :::::: The code at line 966 was first introduced by commit > > > > > > > > > :::::: afe4fd062416b158a8a8538b23adc1930a9b88dc pkt_sched: fq: Fair Queue packet scheduler > > > > > > > > > > > > > > > > > > :::::: TO: Eric Dumazet <edumazet(a)google.com&gt; > > > > > > > > > :::::: CC: David S. Miller <davem(a)davemloft.net&gt; > > > > > > > > > > > > > > > > > > --- > > > > > > > > > 0-DAY CI Kernel Test Service, Intel Corporation > > > > > > > > > https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org > > > > > > > > > _______________________________________________ > > > > > > > > > kbuild mailing list -- kbuild(a)lists.01.org > > > > > > > > > To unsubscribe send an email to kbuild-leave(a)lists.01.org > > > > > > > > > > > > > > > > -- > > > > > > > > You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. > > > > > > > > To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe(a)googlegroups.com. > > > > > > > > To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CANn89i%2Bs%3DoFTq6KF.... > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Thanks, > > > > > > > ~Nick Desaulniers > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe(a)googlegroups.com. > > > > > To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CANn89iLnmKaa30tPoyK_.... > > > > > > > > > > > > > > > > -- > > > > Thanks, > > > > ~Nick Desaulniers > > > > > > > > -- > > Thanks, > > ~Nick Desaulniers > > > > -- > Thanks, > ~Nick Desaulniers -- Thanks, ~Nick Desaulniers