starting up hive within my spring data app

[drosenstark]

I have a script that is running fine when i connect remotely to my hive server but fails with the very generic "Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask, errorCode:1"
when i run the hive server as part of my spring application. I assume that i need to have it start up with same parameters or the like. Has anyone run into this? I have the hive-site.xml on my classpath.

Thanks,
David

[Costin Leau]

hive-site.xml is needed on the server side not the client as most of the configuration in this case is done through Spring. Hive and Hadoop in general are fairly cryptic and there's not much SHDP can do about this.
Make sure to look into the Hive server logs as well to see whether something goes wrong on the server - not having the derby instance running or a missing library tend to be common errors.

[drosenstark]

hive-site.xml is needed on the server side not the client as most of the configuration in this case is done through Spring. Hive and Hadoop in general are fairly cryptic and there's not much SHDP can do about this.
Make sure to look into the Hive server logs as well to see whether something goes wrong on the server - not having the derby instance running or a missing library tend to be common errors.

Just to clarify, my spring data app is trying to boostrap hive. When i run the code such that the client connects to an already running hiveserver, it works fine.When i run the script in the hive CLI it works, but when i run it in the instance created by the xml:
<hdp:hive-server port="${hive.port}" auto-startup="true"
properties-location="hive-server.properties"/>

That is when i get the cryptic errors. So, I am assuming that somehow the defaults that the hive server starts with differ from how it runs when i bring it up at the command line. So, i am wondering how i can have the spring hive instance start exactly as it runs from command line.
My preference is to bring it up in process so we dont have multiple processes accidentally using the same hive server due to warnings about thread safety.

Thanks,
David

[Costin Leau]

Understood. It depends on how you start your hive server by hand - do you rely on any services or configuration? Make sure this are passed properly to hive-server.
Note that by definition, hive-server starts a Thrift server for use with hive-client (a thrift client).
Also, make sure that the hive-conf.xml properties are properly passed to the hive-server - it's best to specify them through properties-location then have them as a file since the classpath can differ. Note that also all the hive-related libraries and dependencies need to be available in the classpath (as opposed to just hive).

[drosenstark]

so i can point to an xml file for the properties instead of a simple properties?
As for libraries, the hive server starts and i include in my classpath the libs of hive as installed on the machine.

[Costin Leau]

no - the properties attributes points to just that, a properties file.
You can however create a dedicated hdp:configuration attribute and pass the XML to that (and potentially set any other properties that you want in a nested fashion):
http://static.springsource.org/sprin...#hadoop:config