Feb 19th, 2013, 08:44 AM
Distributed cache is not working for me with HBase. The mapper reads from hdfs and the reducer writes to HBase. To set-up the conf correctly for the reduce job, I am using the following bean. The initReducerJob() method invokes "TableMapReduceUtil.initTableReducerJob(table, reducer, job);"
<bean id="setupConf4HBase" class="org.springframework.beans.factory.config.Me thodInvokingFactoryBean">
<property name="targetClass"><value>dimension.setup.Initiali zeMRJob</value></property>
The reducer job fails to retrieve files (property files, jars) from DistributedCache. The files are not getting deployed to DistributedCache. I checked the job.xml, there is no trace of these files.
The path.separator is set up properly though. Another non-HBase job defined in the same context file work well and can access files in the DC. In this case (non-HBase job), I can see in job.xml that mapred.cache.files & mapred.job.classpath.files are set-up properly but not in the HBase job.
Any further suggestion or areas to look into?