Thursday, December 19, 2013

How to remove the dead nodes in the Solrcloud manually

Somehow, you may find SolrCloud has a track of all the Nodes even those dead or testing Nodes in the clustermap.

  for example, there is one dead nodes, here
image

If you hover on it, it shows the ip and port.
image

so here we go, remove the 8983 one. Basically the cluster state is keeped in the Zookeeper, all we need to do is download the file and remove the dead/testing nodes, then upload it back. To do this, we need download a full version of the zookeeper , under the bin ,the is a full featured zk client zkCli which enable us to get and set the data.

So save the clusterstate.json to local file.
image

find the dead ndoes, and remove it then save as a new file like new.txt
image

then upload it back,

./zkCli.sh -server 127.0.0.1:9984 set /clusterstate.json "`cat new.txt`"

then it’s gone!~

image

1 comment:

Terry said...

Hi Ryan,
What version of SolrCloud are you using? I have 4.3.1 (can't upgrade unfortunately) and when I try this with a "normal" clusterstate.json file, I get a NumberFormatException with the text of the second line shown as the input. Clearly the version of Zookeeper / Solr's zkcli.sh I have doesn't work the same as yours.

We got around it by taking our clusterstate.json file and removing all newlines and spaces. Then it worked, but looks terrible when viewed in the GUI! Hopefully that will help someone else experiencing a similar issue.

 
Locations of visitors to this page