Saturday, 1 August 2020

External table creation in Hive with Skipping Header line of a fille

// hdfs file has header row in it
hdfs dfs -cat hdfs://localhost:8020/sankara/mysample/mysample.txt
id,name
100,Sara
101,Lara
102,Nila

// Make an external table which will skip the header row from the file
CREATE EXTERNAL TABLE IF NOT EXISTS ohm.tbl_header_removed
(
id int,
name string
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION 'hdfs://localhost:8020/sankara/mysample/'
TBLPROPERTIES("skip.header.line.count"="1");

hive> select * from ohm.tbl_header_removed;
OK
100 Sara
101 Lara
102 Nila
Time taken: 0.715 seconds, Fetched: 3 row(s)

No comments:

Post a Comment

Flume - Simple Demo

// create a folder in hdfs : $ hdfs dfs -mkdir /user/flumeExa // Create a shell script which generates : Hadoop in real world <n>...