SolrCloud installation tutorial (Step-by-Step)

There is no Setup.exe file for Zookeepers. Just unzip the folder „WebOffice Zookeeper“ from zookeeper.zip to the desired location (z.B. C:\Program Files (x86)\VertiGIS\WebOffice Zookeeper).

Unzip WebOffice Zookeeper folder

The WebOffice Zookeeper order contains two instances. If (as we recommend) only one instance should run per host, the instance2 directory must be deleted.

CAUTION!

This must be done individually for each Zookeeper instance.

Custom Zookeeper Instance ID

Each Zookeeper instance of an ensemble has its own integer ID. This ID must be unique across all participating hosts, regardless of how many instances are installed on a single machine.

The ID is stored in the following places:

•The file instance1/data/myid must have the ID as content

•The directory instance1 is renamed to instance{ID}

Configuration of the instance with ID 3

CAUTION!

This must be done individually for each Zookeeper instance.

Customize Zookeeper Server configuration

In the Zookeeper/instance{ID}/conf/zoo.cfg file, the port of the instance and the description of the entire ensemble must be modified.

The list of instances of the quorum is identical for all instances. The list entries are made up as follows:

server.{ID}={Hostname bzw. IP}:{ZK_QUORUM Port}:{ZK_ELECTION Port}

The clientPort (ZK_CLIENT) can be set individually for each instance. However, if all instances run on their own host, it makes sense to use the same ZK_CLIENT everywhere.

Configuration in "zoo.cfg" file

CAUTION!

This must be done individually for each Zookeeper instance.

Customizing the Configuration of the Windows Service

In the file Zookeeper/instance{ID}/conf/wrapper.conf you can configure the name of the Windows service to be started with the Zookeeper.

The value wrapper.ntservice.name is a technical name. If multiple Zookeeper instances are running on the same host, this name must be unique for each instance.

Configuration in "wrapper.conf" file

CAUTION!

This must be done individually for each Zookeeper instance.

Install Windows Service for Zookeepers

On the server, the system variable ZOOKEEPER_HOME must be set to the installation path under Advanced system settings-->Environment Variables.

System variable ZOOKEEPER_HOME

At the Windows command prompt (cmd.exe, with administrator privileges) a Windows service can be created using the script „C:\Program Files (x86)\VertiGIS\WebOffice Zookeeper\instance{ID}\bin\windows-x86-64\InstallNTService.bat“.

Installing and starting the Windows Service

The service can then be started via the script StartNTService.bat or via the list of services.

Precondition for starting this service is an installation of Java on the respective machine.

Windows Service for Zookeepers

CAUTION!

This must be done individually for each Zookeeper instance.

Verify that all instances are included in the Zookeeper Quorum

After all Zookeeper Services have been started, it makes sense to verify that all instances are accessible as expected.

This can be done with the tools netstat.exe, telnet.exe or nc.exe for example.

Additional information can be found in chapter Networktools.

CAUTION!

This must be done individually for each Zookeeper instance.

2. Install SolrCloud

Install WebOffice FTS-Index (one Solr Instance)

An FTS index instance can be installed as usual via the setup (WebOffice_FTS-Index_Setup.msi).

Additional information can be found in the chapter Install WebOffice FTS-Index.

For load balancing, the security setting (IP filtering) of the FTS index instances must be adjusted. Specifically, the IP of the computer on which the LoadBalancer runs must be entered in the file server/etc/jetty-ipfilter.xml by all Solr instances involved so that access is allowed. Instructions can be found in the chapter Configuration of Configuring WebOffice FTS-Index.

CAUTION!

This must be done individually for each Zookeeper instance.

Create root directory in Zookeeper

Zookeeper does not store the configurations to be synchronized on the file system, but keeps them in memory.

With solr.cmd it is possible to list, create or delete directories in Zookeeper.

It is not important to which instance of the Zookeeper Ensemble the command is sent.

List the items in Zookeeper's root:

solr.cmd zk ls / -z localhost:12181

The configurations for Solr should be stored separately from other elements, a new directory /solr is created for this purpose:

solr.cmd zk mkroot /solr -z localhost:12181

Create directory for Solr configurations in Zookeeper

CAUTION!

This must only be done once. The changes are automatically synchronized to the other Zookeeper instances.

Store configuration of the WebOffice collection in Zookeeper

The WebOffice_config folder can be unpacked into a temporary directory (e.g. „C:\Temp\WebOffice_config“) and then uploaded to Zookeeper's memory.

The path of the temporary directory must not contain any special characters or spaces.

The temporarily unpacked data is uploaded with the following command:

solr.cmd zk upconfig -n WebOffice -d C:/Temp/WebOffice_config -z localhost:12181/solr

The -z parameter must be specified in the form [Postname/IP][Port][Root]. The value /solr is decisive, otherwise the uploaded configurations will not end up in the desired directory and will not be found later by the FTS index instances.

Upload the WebOffice configuration to Zookeeper

The ls command can be used to check which configurations are available in Zookeeper:

solr.cmd zk ls /solr -r -z localhost:12181

•The configuration of FTS index is now centrally managed within Zookeeper.

•The local configuration directory WebOffice FTS-Index/server/solr/WebOffice can therefore be deleted. Please note that the Windows service WebOffice FTS Index must be stopped.

•It is not needed in cloud mode and therefore leads at most to confusion.

•The temporary directory (e.g. C:\Temp\WebOffice_config) can also be deleted.

CAUTION!

This must only be done once. The changes are automatically synchronized to the other Zookeeper instances.

Connecting the FTS index to the Zookeeper Ensemble ('Cloud Mode')

In the file FTS-Index/bin/solr.in.cmd the parameter ZK_HOST contains the list of all Zookeeper instances of the ensemble and finally the newly created /solr directory where the configurations should be stored.

set ZK_HOST=localhost:12181,localhost:22181,localhost:32181/solr

Configuration for "Cloud" mode

CAUTION!

Make sure that each individual Zookeeper instance is listed (and not one Zookeeper instance more than once), otherwise failover is not guaranteed.

It is also important to specify the path /solr at the end so that the uploaded configurations can be found.

The Windows service WebOffice FTS Index (port 8983) must now be restarted and is from then on in Cloud mode. This can be seen from the Cloud menu item available in the Solr web interface.

Also, no collections or cores are defined, since the configuration is now obtained from Zookeeper.

Solr WebInterface in "Cloud"-Mode

CAUTION!

This must be done individually for each installed WebOffice FTS-Index instance.

The final configuration depends on whether you are using WebOffice FTS-Index in an http or https scenario.

Continue at point B) if the configuration is to be done in an https scenario.

A) Create Solr Collection WebOffice and distribute it to all FTS instances (http-scenario)

Now a new collection is created from the already uploaded configuration set and this is replicated to all available AGV instances.

solr.cmd create -p 8983 -c WebOffice -n WebOffice -shards 1 -replicationFactor 2

Creation of the Collection WebOffice on two available Solr instances

The warning „WARNING: Using _default configset …“ appears with a bug in Solr and can be ignored if the expected messages Re-using existing configuration ... and Created collection... are displayed afterwards.

CAUTION!

This must be done ONLY ONCE. The changes will be automatically synchronized to the other Zookeeper instances.

In the Solr WebInterface you can check under the menu item Cloud whether the creation was successful:

http://localhost:8983/solr/#/~cloud

"Nodes" menu item: Shows all involved Solr instances

The collection "WebOffice" is replicated to two Solr nodes

B) Create Solr Collection WebOffice and distribute it to all FTS instances (https-scenario)

This requires a configured encrypted connection via SSL/TLS, which is described in the chapter Configuring WebOffice FTS-Index.

All involved Zookeeper host names or IP addresses must be listed as Subject Alternative Name (SAN) in the used SSL/TLS certificate. Otherwise, the SSL handshake cannot be performed successfully.

In case of https communication between the Zookeeper instances the following command must be issued in the command prompt (CMD):

WebOffice FTS-Index\server\scripts\cloud-scripts\zkcli.bat -zkhost server1:12181,server2:12181,server3:12181/solr -cmd clusterprop -name urlScheme -val https

wo_fts_application_cloud_install15

For more details see also Apache Solr Reference Guide.

Now a new collection can be created from the already uploaded configuration set and this will be replicated to all available FTS instances.

WebOffice FTS-Index\bin\solr.cmd create -p 8983 -c WebOffice -n WebOffice -shards 1 -replicationFactor 2

Creation of the Collection WebOffice on two available Solr instances

CAUTION!

This must be done ONLY ONCE. The changes will be automatically synchronized to the other Zookeeper instances.

In the Solr WebInterface you can check under the menu item Cloud whether the creation was successful:

https://localhost:8983/solr/#/~cloud

"Nodes" menu item: Shows all involved Solr instances

The collection "WebOffice" is replicated to two Solr nodes