View Full Version : Is it possible to inject properties/beans into mapper/reducer instances ?
vito
Apr 10th, 2012, 03:54 AM
Hi,
I need to use some beans within a mapper context.
What I have used so far is serilaise the
Is spring IOC invoked with each hadoop child process?
If it is, how can I declare it, The xsd (http://static.springsource.org/spring-hadoop/docs/current/reference/html/appendix-schema.html) permitts only class FQN
Thanks,
Costin Leau
Apr 12th, 2012, 06:42 AM
We have (on a branch) support for mapper/reduce instance configuration if you'd like to experiment with that. Currently we don't support that in the main trunk but we do plan to merge it at some point.
The main downside of instance configuration is that each node needs to recreate the wiring which usually adds some overhead for provisioning (each note for example needs to have the support classes present). We have some ideas on how to mitigate that but we haven't gotten around it yet.
Cheers,
Loke
Jul 28th, 2012, 07:45 AM
Has this been added to the master branch?
Costin Leau
Jul 29th, 2012, 11:41 AM
We have (on a branch) support for mapper/reduce instance configuration if you'd like to experiment with that. Currently we don't support that in the main trunk but we do plan to merge it at some point.
...
See above.
hsn
Aug 7th, 2012, 05:33 AM
code is here
https://github.com/SpringSource/spring-hadoop/tree/mr-pojo/src/main/java/org/springframework/data/hadoop/context
Powered by vBulletin® Version 4.2.1 Copyright © 2013 vBulletin Solutions, Inc. All rights reserved.