Setup solr using zookeeper ensemble on ubnutu

solrcould-cluster-single-collection-zookeeper-ensemble

Setup Oracle Java:
Follow quick step given below to setup java latest version on your system,

java -version
tar -zxvf jdk-8u45-linux-x64.tar.gz
sudo mkdir -p /usr/lib/jvm/jdk1.8.0_45
sudo mv jdk1.8.0_45/* /usr/lib/jvm/jdk1.8.0_45/
sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.8.0_45/bin/java" 1
sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.8.0_45/bin/javac" 1
sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.8.0_45/bin/javaws" 1
sudo update-alternatives --set java /usr/lib/jvm/jdk1.8.0_45/bin/java
sudo update-alternatives --set javac /usr/lib/jvm/jdk1.8.0_45/bin/javac
sudo update-alternatives --set javaws /usr/lib/jvm/jdk1.8.0_45/bin/javaws
java -version
vim ~/.bashrc
export JAVA_HOME="/usr/lib/jvm/jdk1.8.0_45"
export PATH="$PATH:$JAVA_HOME/bin"
source ~/.bashrc
echo $JAVA_HOME
echo $PATH

For more detail kindly visit how-to-install-oracle-jdk-7-on-ubuntu-12-04

Setup Zookeeper Ensemble: (https://cwiki.apache.org/confluence/display/solr/Setting+Up+an+External+ZooKeeper+Ensemble)

Consider we have 2 servers (192.168.0.101, 192.168.0.111) and we are setting up 6 zookeeper nodes called znodes.
Follow quick step given below to setup Zookeeper Ensemble on your system,

wget 'http://mirror.symnds.com/software/Apache/zookeeper/stable/zookeeper-3.4.6.tar.gz'
tar -xvf zookeeper-3.4.6.tar.gz
sudo mkdir -p /usr/lib/zookeeper-3.4.6
sudo mv zookeeper-3.4.6/* /usr/lib/zookeeper-3.4.6/
cd /usr/lib/zookeeper-3.4.6/
sudo cp conf/zoo_sample.cfg conf/zoo_1.cfg
sudo vim conf/zoo_1.cfg

Add following configs,

tickTime=2000
dataDir=/var/lib/zookeeper/1/
clientPort=2181
initLimit=5
syncLimit=2
server.1=192.168.0.101:2888:3888
server.2=192.168.0.101:2889:3889
server.3=192.168.0.101:2890:3890
server.4=192.168.0.111:2888:3888
server.5=192.168.0.111:2889:3889
server.6=192.168.0.111:2890:3890

Repeat above setting for all servers.

Make data directories:(Repeat for all servers)

sudo mkdir -p /var/lib/zookeeper/1/
sudo sh -c 'echo "1" > /var/lib/zookeeper/1/myid'

Start servers:(Repeat for all servers)
sudo bin/zkServer.sh start zoo_1.cfg

Check Status:

bin/zkServer.sh status zoo_1.cfg
echo status | nc localhost 2187

Test Client:

bin/zkCli.sh -server 192.168.0.101:2181
[zk: 192.168.0.101:2181(CONNECTED) 1] ls /
[zk: 192.168.0.101:2181(CONNECTED) 2] get /configs/gettingstarted/solrconfig.xml
[zk: 192.168.0.101:2181(CONNECTED) 3] quit

Stop Servers:(Repeat for all servers)
sudo bin/zkServer.sh stop zoo_1.cfg

For more detail kindly visit Setup-zookeeper-ensemble-on-ubuntu

Setup Solr:

Consider we have 2 servers (192.168.0.101, 192.168.0.111) and we are setting up 2 solr instances in shard=1:replication:2 setup.
wget 'http://apache.mirrors.hoobly.com/lucene/solr/5.2.0/solr-5.2.0.tgz'
SolrCloud Standalone Setup using Embedded Zookeeper (Testing environment): (http://lucene.apache.org/solr/quickstart.html)
Please follow above link for demos.

SolrCloud Mode Setup using External Zookeeper Ensemble (Testing environment): (https://cwiki.apache.org/confluence/display/solr/Solr+Start+Script+Reference)
cd solr-5.2.0/tar -zxvf solr-5.2.0.tgz

Create Node:

mkdir -p example/cloud/node1/solr
cp server/solr/solr.xml example/cloud/node1/solr
mkdir -p example/cloud/node2/solr
cp server/solr/solr.xml example/cloud/node2/solr
bin/solr start -cloud -s example/cloud/node1/solr -h 192.168.0.101 -p 8983 -z 192.168.0.101:2181,192.168.0.101:2182,192.168.0.101:2183,192.168.0.111:2184,192.168.0.111:2185,192.168.0.111:2186
bin/solr start -cloud -s example/cloud/node1/solr -h 192.168.0.111 -p 8983 -z 192.168.0.101:2181,192.168.0.101:2182,192.168.0.101:2183,192.168.0.111:2184,192.168.0.111:2185,192.168.0.111:2186

Create Collection:
bin/solr create -c gettingstarted -d basic_configs -rf 2

Status:

bin/solr status
bin/solr healthcheck -c gettingstarted

Delete Collection:
bin/solr delete -c gettingstarted

Restart:

bin/solr restart -cloud -s example/cloud/node1/solr -h 192.168.0.101 -p 8983 -z 192.168.0.101:2181,192.168.0.101:2182,192.168.0.101:2183,192.168.0.111:2184,192.168.0.111:2185,192.168.0.111:2186
bin/solr restart -cloud -s example/cloud/node1/solr -h 192.168.0.111 -p 8983 -z 192.168.0.101:2181,192.168.0.101:2182,192.168.0.101:2183,192.168.0.111:2184,192.168.0.111:2185,192.168.0.111:2186

Stop Node:
bin/solr stop -all;

Clean all Testing files:
rm -Rf example/cloud/

SolrCloud as a Service using External Zookeeper Ensemble (Production environment): (https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production)

Run Installation script:

tar xzf solr-5.2.0.tgz solr-5.2.0/bin/install_solr_service.sh --strip-components=2
sudo bash ./install_solr_service.sh -help
sudo bash ./install_solr_service.sh solr-5.0.0.tgz -i /opt -d /var/solr -u solr -s solr -p 8983
OR
sudo bash ./install_solr_service.sh solr-5.2.0.tgz
id: solr: no such user
Creating new user: solr
Adding system user `solr' (UID 109) ...
Adding new group `solr' (GID 116) ...
Adding new user `solr' (UID 109) with group `solr' ...
Creating home directory `/home/solr' ...
Extracting solr-5.2.0.tgz to /opt
Creating /etc/init.d/solr script ...
Adding system startup for /etc/init.d/solr ...
Waiting to see Solr listening on port 8983 [/]
Started Solr server on port 8983 (pid=1704). Happy searching!
Service solr installed.
sudo service solr status
sudo service solr stop

Setup SolrCloud:

To run Solr in SorlCloud add following setting in environment specific include file (/var/solr/solr.in.sh),
ZK_HOST="192.168.0.101:2181,192.168.0.101:2182,192.168.0.101:2183,192.168.0.111:2184,192.168.0.111:2185,192.168.0.111:2186"

If you’re using a ZooKeeper instance that is shared by other systems, it’s recommended to isolate the SolrCloud znode tree using ZooKeeper’s chroot support. For instance, to ensure all znodes created by SolrCloud are stored under /solr, you can put /solr on the end of your ZK_HOST connection string, such as:
ZK_HOST="192.168.0.101:2181,192.168.0.101:2182,192.168.0.101:2183,192.168.0.111:2184,192.168.0.111:2185,192.168.0.111:2186/solr"

If using a chroot for the first time, you need to bootstrap the Solr znode tree in ZooKeeper by using the zkcli.sh script, such as:
/opt/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost 192.168.0.101:2181 -cmd bootstrap -solrhome /var/solr/data

If above script couldn’t be able to create ‘solr’ chroot because of 0 core (As we have not create any core yet) then create one by zookeeper client,

/usr/lib/zookeeper-3.4.6/bin/zkCli.sh -server 192.168.0.101:2181
[zk: 192.168.0.101:2181(CONNECTED) 2] create /solr solr

Note: Above fix is not specified anywhere in Solr Doc.
sudo service solr start

Upload a configuration directory: (https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities)
/opt/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost 192.168.0.101:2181/solr -cmd upconfig -confname data_driven_schema_configs -confdir /opt/solr/server/solr/configsets/data_driven_schema_configs/conf

To delete wrong config use zookeeper client,

/usr/lib/zookeeper-3.4.6/bin/zkCli.sh -server 192.168.0.101:2181
[zk: 192.168.0.111:2184(CONNECTED) 8] rmr /configs

Create Collection:(https://cwiki.apache.org/confluence/display/solr/Collections+API)

http://host:port/solr/admin/collections?action=CREATE&name=gettingstarted&numShards=1&replicationFactor=1&maxShardsPerNode=1&collection.configName=data_driven_schema_configs

http://host:port/solr/admin/collections?action=DELETE&name=gettingstarted

For more detail kindly visit Install-solr-on-ubuntu

Important Download links,

Search Engine research:

https://www.elastic.co/products/elasticsearch

http://solr-vs-elasticsearch.com/

http://lucene.apache.org/solr/quickstart.html

https://cwiki.apache.org/confluence/display/solr/Apache+Solr+Reference+Guide

http://wiki.apache.org/solr/FrontPage

https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler

Requirement:

http://lucene.apache.org/solr/5_1_0/SYSTEM_REQUIREMENTS.html

Downloads:

http://mirror.reverse.net/pub/apache/lucene/solr/5.1.0/solr-5.1.0.tgz

http://apache.mirrors.hoobly.com/lucene/solr/5.2.0/solr-5.2.0.tgz

http://download.oracle.com/otn-pub/java/jdk/8u45-b14/jdk-8u45-linux-x64.tar.gz

http://archive.apache.org/dist/lucene/solr/

http://mirror.symnds.com/software/Apache/zookeeper/stable/zookeeper-3.4.6.tar.gz

Setup:

https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production

http://zookeeper.apache.org/doc/r3.4.6/zookeeperStarted.html

PHP Client:

http://wiki.apache.org/solr/SolPHP

https://pecl.php.net/package/solr

http://php.net/manual/en/book.solr.php

Symfony Bundle:

https://packagist.org/packages/solarium/solarium

https://packagist.org/packages/nelmio/solarium-bundle

https://packagist.org/packages/reprovinci/solr-php-client

https://packagist.org/packages/floriansemm/solr-bundle

https://packagist.org/packages/internations/solr-utils

https://packagist.org/packages/internations/solr-query-component


Thank you.

This entry was posted in Java, Solr, Uncategorized, Zookeeper and tagged , , . Bookmark the permalink.

Leave a Reply