create a directory in hdfs:
hdfs dfs -mkdir /user/dir1
hdfs dfs -ls /user
create an empty file in local file system:
touch test_file11.txt
copy the local file into hdfs: (upload a file to hdfs)
hdfs dfs -put /root/test_file11.txt /user/dir1
hdfs dfs -ls /user/dir1
create a directory in hdfs:
hdfs dfs -mkdir /user/dir2
hdfs dfs -ls /user
copy a file within hdfs: dir1 to dir2:
hdfs fs -cp /user/dir1/test_file11.txt /user/dir2
hdfs fs -ls /user/dir2
fsck : to display the summary of file system health
hdfs fsck /user/dir1/sample_data/salary/salary.csv
file is HEALTHY
Missing blocks.
To know the Size of a given file:
hdfs dfs -du /user/sample_data/salary/salary.csv
Remove file(s) in hdfs:
hdfs dfs -rm /user/dir2/*
Download a file from hdfs to local directory:
hdfs dfs -get /user/sample_data/salary/salary.csv /root/
rm /root/salary.csv --> remove local file
hdfs dfs -get /user/sample_data/salary/salary.csv /root/
Delete existing things:
hdfs dfs -rm -r /user/*
Create Directory:
hdfs dfs -mkdir /user
Create Directories:
hdfs dfs -mkdir /user/dir1 /user/dir2
List directory:
hdfs dfs -ls /
hdfs dfs -ls /user
[cloudera@quickstart ~]$ hdfs dfs -ls /user
Found 2 items
drwxr-xr-x - cloudera cloudera 0 2018-07-13 22:48 user/dir1
drwxr-xr-x - cloudera cloudera 0 2018-07-13 22:48 user/dir2
Create an empty file inside ./user/dir1
hdfs dfs -touchz /user/dir1/my1st.txt
list it again
hdfs dfs -ls /user/dir1
Found 1 items
-rw-r--r-- 1 cloudera cloudera 0 2018-07-13 22:50 user/dir1/my1st.txt
Remove a file:
hdfs dfs -rm /user/dir1/my1st.txt
Deleted user/dir1/my1st.txt
Remove Directory:
hdfs dfs -rmdir /user/dir1 /user/dir1
Create a file in local linux:
cat > salary.csv
Name,Age,Sex,Salary
Arun,25,M,2000
Balu,30,M,2500
Sara,42,M,2050
Lara,40,F,1980
^c (Ctrl + c)
Display the content of file in local linux:
cat salary.csv
Name,Age,Sex,Salary
Arun,25,M,2000
Balu,30,M,2500
Sara,42,M,2050
Lara,40,F,1980
Display the present working directory in local linux:
pwd
/home/cloudera
Display the files, folders in local linux:
ls -l
total 192
-rwxrwxr-x 1 cloudera cloudera 5387 Jul 19 2017 cloudera-manager
-rwxrwxr-x 1 cloudera cloudera 9964 Jul 19 2017 cm_api.py
drwxrwxr-x 3 cloudera cloudera 4096 Jul 13 22:43 Desktop
drwxrwxr-x 4 cloudera cloudera 4096 Jul 19 2017 Documents
drwxr-xr-x 2 cloudera cloudera 4096 Sep 5 2017 Downloads
drwxrwsr-x 9 cloudera cloudera 4096 Feb 19 2015 eclipse
-rw-rw-r-- 1 cloudera cloudera 53655 Jul 19 2017 enterprise-deployment.json
-rw-rw-r-- 1 cloudera cloudera 50515 Jul 19 2017 express-deployment.json
-rwxrwxr-x 1 cloudera cloudera 5007 Jul 19 2017 kerberos
drwxrwxr-x 2 cloudera cloudera 4096 Jul 19 2017 lib
drwxr-xr-x 2 cloudera cloudera 4096 Sep 5 2017 Music
-rwxrwxr-x 1 cloudera cloudera 4228 Jul 19 2017 parcels
drwxr-xr-x 2 cloudera cloudera 4096 Sep 5 2017 Pictures
drwxr-xr-x 2 cloudera cloudera 4096 Sep 5 2017 Public
-rw-rw-r-- 1 cloudera cloudera 80 Jul 13 22:57 salary.csv /// we recently created it
drwxr-xr-x 2 cloudera cloudera 4096 Sep 5 2017 Templates
drwxr-xr-x 2 cloudera cloudera 4096 Sep 5 2017 Videos
drwxrwxr-x 4 cloudera cloudera 4096 Jul 19 2017 workspace
Create a folder in hdfs:
hdfs dfs -mkdir /user/dir1
[cloudera@quickstart ~]$ hdfs dfs -ls /user/
Found 2 items
drwxr-xr-x - cloudera cloudera 0 2018-07-13 23:11 user/dir1
Display the content a file which is in local linux:
cat /home/cloudera/salary.csv
Name,Age,Sex,Salary
Arun,25,M,2000
Balu,30,M,2500
Sara,42,M,2050
Lara,40,F,1980
Upload file:
hdfs dfs -put <local file> <hdfs dest>
hdfs dfs -put /home/cloudera/salary.csv /user/dir1/salary.csv
hdfs dfs -put /home/cloudera/apple_daily.csv /user/dir1/apple_daily.csv
Display the content of a file which is in hdfs:
hdfs dfs -cat /user/dir1/salary.csv
Name,Age,Sex,Salary
Arun,25,M,2000
Balu,30,M,2500
Sara,42,M,2050
Lara,40,F,1980
downlaod file:
hdfs dfs -get <hdfs source> <local dest>
hdfs dfs -get /user/dir1/salary.csv /home/cloudera/mysal.csv
Display the content of downloaded file:
cat /home/cloudera/mysal.csv
Name,Age,Sex,Salary
Arun,25,M,2000
Balu,30,M,2500
Sara,42,M,2050
Lara,40,F,1980
File size:
hdfs dfs -du <filename>
hdfs dfs -du /user/dir1/salary.csv
80 80 user/dir1/salary.csv
Delete file:
hdfs dfs -rm /user/dir1/salary.csv
Deleted user/dir1/salary.csv
get Help:
hdfs dfs -help
Create zero size file:
hdfs dfs -touchz /user/dir1/sa.txt
delete it
hdfs dfs -rm /user/dir1/sa.txt
copy file within hdfs folders:
hdfs dfs -cp /user/dir1/my.txt /user/dir2/my2.txt
hdfs dfs -ls /user/dir2 :: see it here
hdfs fsck /user/dir2
hadoop fs -du /user/dir1/apple.csv
hdfs dfs -du /user/dir1/apple.csv
hdfs dfs -ls
hdfs dfs -ls /user/
hdfs dfs -cat /user/hue/my.csv
hadoop fs -cat /user/hue/my.csv
hdfs dfs -help
hdfs dfs -ls
hdfs dfs -touchz <file> : creates file with zero length.
hdfs dfs -cat <file> :: displays the file content to the screen
hdfs dfs -mkdir /newfolder :: create directory / folder
hdfs dfs -cp <source file> <target file> /// within hdfs
hdfs dfs -mv <old file> <new file> :: different directory can be
hdfs dfs -mv /user/dir1/apple_daily.csv /user/dir1/apple.csv
hdfs dfs -help mkdir
hdfs dfs -ls
hdfs dfs -ls /user/
hdfs dfs -ls /user/hue
hdfs dfs -cat /user/hue/my.csv
hdfs dfs -cp /user/hue/my.csv /user/hue/ours.csv
hdfs dfs -mkdir /user/hue/hdfs-course
hdfs dfs -put apple.csv /user/hue/hdfs-course/test.csv
hdfs dfs -mv /user/hue/hdfs-course/test.csv awe.csv
hdfs dfs -put /tmp/file /user/hue/file
hdfs dfs -get /user/hdfs/file /tmp/file
hdfs dfs -mkdir /user/thehenson
hdfs dfs -moveFromLocal apple.csv /user/thehenson/appledaily.csv
hdfs dfs -get /user/thehenson/appledaily.csv /tmp/apple.csv
hdfs dfs -mkdir /user/thehenson
Empty the recycle bin : hdfs dfs -expunge
hdfs file location :
/user/sample_data/salary/
hdfs fsck /
hdfs dfs –ls /
hdfs dfs –mkdir /new_edureka
hdfs dfs –touchz /new_edureka/sample
hdfs dfs –du –s /new_edureka/sample
hdfs dfs –cat /new_edureka/test
hdfs dfs –copyFromLocal /home/edureka/test /new_edureka
hdfs dfs –copyToLocal /new_edureka/test /home/edureka
hdfs dfs –put /home/edureka/test /user
hdfs dfs –get /user/test /home/edureka
hdfs dfs –count /user
hdfs dfs –rm /new_edureka/test
hdfs dfs -rm -r /new_edureka (remove folder and its contents everything)
hdfs dfs -cp /user/hadoop/file1 /user/hadoop/file2 /user/hadoop/dir
hdfs dfs -mv /user/hadoop/file1 /user/hadoop/file2
hdfs dfs -expunge --> Trash empty
hdfs dfs –rmdir /user/hadoop
hdfs dfs -usage mkdir
hdfs dfs -help
hadoop namenode -format
hadoop secondrynamenode
hadoop namenode
No comments:
Post a Comment