Monday, 28 January 2019

How to hide warning, Info messages in Spark-shell?

// Hide INFO messages in spark-shell
import org.apache.log4j.Logger
import org.apache.log4j.Level

Logger.getLogger("org").setLevel(Level.OFF)
Logger.getLogger("akka").setLevel(Level.OFF)

-
Labels:

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Flume - Simple Demo

// create a folder in hdfs : $ hdfs dfs -mkdir /user/flumeExa // Create a shell script which generates : Hadoop in real world <n>...