This article is giving you information about the installation of spark single node master slaves cluster formation steps. Follow this and get a Master Slaves spark cluster in your machine and enjoy it.
1. Download spark from the below link. If you are using linux os then just use the below wget command to get the spark version. If you need latest version refer official Spark Site
2. To start spark in your machine we need scala. To download scala use the below wget command. And wait untill scala gets downloaded in your machine. If you need latest version of scala enter in to official scala Site
3. After getting the spark-1.3.0.tgz and scala-2.10.4.tgz on your machine just untar both the file.
$tar -zxvf spark-1.3.0.tar.gz
$tar -zxvf scala-2.10.4.tar.gz
After untaring set the scala and spark path on your .bashrc and source the bashrc file.
Set in etc/hosts
$sudo vi /etc/hosts
Now we need to install git
sudo apt-get install git
cp spark-env.sh.template spark-env.sh
$cp slaves.template slaves
ipaddress (datadotz_master (or) 10.0.0.7)
After completing these configurations now enter in to spark-1.3.0 folder in terminal and give the following commands to start master and slave
Article written by DataDotz Team
DataDotz is a Chennai based BigData Team primarily focussed on consulting and training on technologies such as Apache Hadoop, Apache Spark , NoSQL(HBase, Cassandra, MongoDB), Search and Cloud Computing.
Note: DataDotz also provides classroom based Apache Kafka training in Chennai. The Course includes Cassandra , MongoDB, Scala and Apache Spark Training. For more details related to Apache Spark training in Chennai, please visit http://datadotz.com/training/