is there any mentrics to show the usage of executor on memory or CPU

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

is there any mentrics to show the usage of executor on memory or CPU

zhangliyun
Hi all:
  i want to ask a question about the metrics to show the executor is fully used the memory.  in the log i always saw following  in the log, i guess this means i did not fully use the executor 's memory.
 but i don't want to open the log to view, is there any metrics to show it?

my config:
SPARK_EXECUTOR_MEMORY==16G
SPARK_EXECUTOR_CORE=2

20/05/15 02:39:01 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on lvshdc5dn3718.lvs.paypalinc.com:45488 (size: 42.7 KB, free: 4.1 GB)

20/05/15 02:39:01 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on lvshdc5dn4353.lvs.paypalinc.com:35598 (size: 42.7 KB, free: 4.1 GB)

20/05/15 02:39:01 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on lvshdc5dn3847.lvs.paypalinc.com:35152 (size: 42.7 KB, free: 4.1 GB)


my question is "is there any mentrics to show the usage of executor on memory or CPU" , appreciate if you have any suggestion

 Best Regards
Kelly Zhang