Apache Hadoop 1.0.0.RC1 (SHDP) : Getting class cast error while executing Batch Jobs:
Hi,
I am new to SHDP and have been trying to build the environment for batch execution of Jobs.
However, with the use of the eclipse Plugin (Map/reduce) to run the job using SHDP, I am getting the below error:
I see the context initialization happening properly, however, I get the class casting exception from map-reduce job - HDP to spring job.
Please refer the error log below: -
Dec 11, 2012 9:23:41 AM org.springframework.context.support.AbstractApplic ationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlAp plicationContext@9a9b65: startup date [Tue Dec 11 09:23:41 IST 2012]; root of context hierarchy
Dec 11, 2012 9:23:41 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [resources/launch-context.xml]
Dec 11, 2012 9:23:43 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [META-INF/spring/batch-common.xml]
Dec 11, 2012 9:23:43 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [META-INF/spring/hadoop-context.xml]
Dec 11, 2012 9:23:43 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [META-INF/spring/wordcount-context.xml]
Dec 11, 2012 9:23:44 AM org.springframework.core.io.support.PropertiesLoad erSupport loadProperties
INFO: Loading properties file from class path resource [batch.properties]
Dec 11, 2012 9:23:44 AM org.springframework.core.io.support.PropertiesLoad erSupport loadProperties
INFO: Loading properties file from class path resource [hadoop.properties]
Dec 11, 2012 9:23:44 AM org.springframework.beans.factory.support.DefaultL istableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultL istableBeanFactory@c51ce4: defining beans [org.springframework.beans.factory.config.PropertyP laceholderConfigurer#0,org.springframework.context .annotation.internalConfigurationAnnotationProcess or,org.springframework.context.annotation.internal AutowiredAnnotationProcessor,org.springframework.c ontext.annotation.internalRequiredAnnotationProces sor,org.springframework.context.annotation.interna lCommonAnnotationProcessor,jobRepository,transacti onManager,jobLauncher,hadoopConfiguration,org.spri ngframework.data.hadoop.scripting.HdfsScriptRunner #0,script-tasklet,wordcount-job,org.springframework.context.annotation.Configu rationClassPostProcessor.importAwareProcessor]; root of factory hierarchy
Dec 11, 2012 9:23:44 AM org.springframework.batch.core.launch.support.Simp leJobLauncher afterPropertiesSet
INFO: No TaskExecutor has been set, defaulting to synchronous executor.
Dec 11, 2012 9:23:44 AM org.apache.hadoop.conf.Configuration <clinit>
WARNING: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Checking jobName [org.apache.hadoop.mapreduce.Job@1ee89c0]
Executing job wordcount-job
Exception in thread "main" org.springframework.beans.factory.BeanInitializati onException: Cannot execute job wordcount-job; nested exception is java.lang.ClassCastException: org.apache.hadoop.mapreduce.Job cannot be cast to org.springframework.batch.core.Job
at org.apache.hadoop.jobManager.JobManager.startJobMa nager(JobManager.java:46)
at org.springframework.data.hadoop.samples.wordcount. Main.main(Main.java:42)
Caused by: java.lang.ClassCastException: org.apache.hadoop.mapreduce.Job cannot be cast to org.springframework.batch.core.Job
at org.apache.hadoop.jobManager.JobManager.startJobMa nager(JobManager.java:42)
... 1 more
