Pig, HBase and Spring Data Hadoop

[BinaryTweedDeej]

Has anyone got experience of using Spring Data Hadoop to run a Pig script that connects to HBase using Elephant Bird's HBaseLoader?

I'm new to all of the above, but need to take some existing Pig scripts that were executed via a shell script and instead wrap them up in a self-contained Java application. Currently the scripts are run from a specific server that has Hadoop, HBase and Pig installed, and config for all of the above in /etc/. Pig has the HBase config on its classpath, so I'm guessing this is how it know how to connect to HBase

I want to have all configuration in Spring. Is this possible if I need Pig to connect to HBase? How do I configure HBase such that the Pig script and the Elephant Bird library will know how to connect to it?

[Costin Leau]

Haven't tried this myself but I suspect Elephant Bird library would be registered in Pig through a script (which SHDP supports). Not sure what the HBase connectivity implies - I suspect another Pig function/library that gets loaded which you could parameterize so the hbase location/port are not hardcoded.

[BinaryTweedDeej]

Hi Costin,

Thanks for your reply. As you may have guessed, my understanding of the Hadoop ecosystem is a little fuzzy!

I have recently discovered what I think to be an unrelated bug in 1.0.0.M1: https://jira.springsource.org/browse/SHDP-38