i tried install , build spark 2.0.0 on ubuntu vm ubuntu 16.04 follows:
install java
sudo apt-add-repository ppa:webupd8team/java sudo apt-get update sudo apt-get install oracle-java8-installer
install scala
go downloads tab on site: scala-lang.org/download/all.html
i used scala 2.11.8.
sudo mkdir /usr/local/src/scala sudo tar -xvf scala-2.11.8.tgz -c /usr/local/src/scala/
modify
.bashrc
file , include path scala:export scala_home=/usr/local/src/scala/scala-2.11.8 export path=$scala_home/bin:$path
then type:
. .bashrc
install git
sudo apt-get install git
download , build spark
go to: http://spark.apache.org/downloads.html
download spark 2.0.0 (build source - standalone mode).
tar -xvf spark-2.0.0.tgz cd spark folder (that has been extracted).
now type:
./build/sbt assembly
after done installing, message:
[success] total time: 1940 s, completed...
followed date , time...
run spark shell
bin/spark-shell
that's when hell breaks loose , start getting error. go assembly folder folder called target. there's no such folder there. things visible in assembly are: pom.xml, readme, , src.
i looked online quite while , haven't been able find single concrete solution solve error. can please provide explicit step-by-step instructions how go solving ?!? it's driving me nuts now... (t.t)
screenshot of error:
for reason, scala 2.11.8 not working while building if switch on scala 2.10.6 builds properly. guess reason need scala in first place access sbt able build spark. once built, need direct myself spark folder , type:
build/sbt package
this build missing jar files me using scala 2.11... kinda weird that's how working (i assuming looking @ logs).
once spark builds again, type: bin/spark-shell (while being in spark folder) , you'll have access spark shell.
Comments
Post a Comment