Tuesday, April 30, 2013

Hadoop "Incompatible namespaceIDs" error

This is an error that I faced a while ago and I made a blog entry to be publised some day, so here it goes.

When setting up an installation and if not everything goes well, we might face the situation of datanode/namenode desyncrhonization. You should immediately refer to the logs. There you can see the following message:
 ... ERROR org.apache.hadoop.dfs.DataNode: java.io.IOException: Incompatible namespaceIDs in /app/hadoop/tmp/dfs/data: namenode namespaceID = 308967713; datanode namespaceID = 113030094
        at org.apache.hadoop.dfs.DataStorage.doTransition(DataStorage.java:281)
        at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.java:121)
  1. Go to you namenode's directory (dfs.name.dir) and get the namespaceID value from current/VERSION.
  2. Go to your data node's storage directory (dfs.data.dir) and edit current/VERSION.
  3. Change the value to the datanode's value for the namenode's value.


I had not been able to repair the filesystem with any other way.