[Hadoop] Windows 환경에서 실행시 오류 해결

[업데이트 2017.02.10 15:39]

 

1. JAVA_HOME 오류

HADOOP_HOME/etc/hadoop/hadoop-env.cmd 수정.(Program Files space때문에 오류 발생)

symbolic link로 지정함.

mklink /D \java_home "C:\Program Files\Java\jre1.8.0_111"
set JAVA_HOME=\java_home

 

* 참고: http://stackoverflow.com/questions/26990243/hadoop-installation-on-windows

 

2. util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable.. 오류

Windows환경에서 Hadoop node 구축시 발생하는 오류중에 아래의 오류가 있는데, winutils.exe를 HADOOP_HOME/bin폴더에 복사후 실행하면 오류가 해결 되었습니다. 아래 내용 참고하시기 바랍니다.

2014-04-17 16:22:08,599 ERROR [main] util.Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
    at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:278)
    at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:300)
    at org.apache.hadoop.util.Shell.(Shell.java:293)
    at org.apache.hadoop.util.StringUtils.(StringUtils.java:76)
    at org.apache.hadoop.conf.Configuration.getStrings(Configuration.java:1514)
    at org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:113)
    at org.apache.hadoop.hbase.zookeeper.ZKServerTool.main(ZKServerTool.java:46)

 

정리하면,

 

1. winutils.exe를 HADOOP_HOME/bin폴더에 복사.(http://public-repo-1.hortonworks.com/hdp-win-alpha/winutils.exe)

2. HADOOP_HOME environment variable 설정.

ex) HADOOP_HOME=C:\hadoop-2.7.3

 

* 참고: https://wiki.apache.org/hadoop/Hadoop2OnWindows

* 참고: http://stackoverflow.com/questions/35652665/java-io-ioexception-could-not-locate-executable-null-bin-winutils-exe-in-the-ha

* 참고: http://stackoverflow.com/questions/23136092/hadoophbase-cluster-on-windows-winutils-not-found