Feb 28th, 2013, 08:32 AM
FileSystem closed error in hadoop 2.0.3-alpha
In hadoop 2, FileSystem for each type is singleton. Calling FileSystem.close() closes it for all future calls.
After updating from RC1 to GA, i can not run tasks working with @Inject filesystem bean because fs is closed.
I was able to discover only changes to HdfsResouceLoader which now closes Filesystem. I changed code to call it with filesystem instance to avoid closing fs, but it is still not enough to get it running.
my code is using: ConfigurationFactoryBean, HdfsResourceLoader, DistributedCacheFactoryBean.
Feb 28th, 2013, 08:43 AM
Problem is in DistributedCacheFactoryBean. It creates HDFSResourceLoader from configuration, instead of filesystem. Closing its internal resourceloader closes filesystem as well, causing hadoop 2.0 drivers (Tool) to stop working.