Possible issue with closing connections in HBaseTemplate?

**michaeldowling** · Jul 11th, 2012, 08:05 PM

Hey all,

I'm hoping you can help me with something that is perplexing me. I'm a bit of an HBase newb (and certainly spring-data-hadoop newb).

I have the following configuration set in my spring config:

Code:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:hdp="http://www.springframework.org/schema/hadoop"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                                 http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd">

    <hdp:hbase-configuration id="hbaseConfig" />
    <bean id="hbaseTemplate" class="org.springframework.data.hadoop.hbase.HbaseTemplate">
        <property name="configuration" ref="hbaseConfig" />
    </bean>


</beans>

I am injecting the template into my own DAO class:

Code:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:hdp="http://www.springframework.org/schema/hadoop"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                                 http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd">



    <bean id="userStore" class="data.store.hbase.UserStoreHBaseImpl">
        <constructor-arg name="hbaseTemplate" ref="hbaseTemplate" />
    </bean>

</beans>

My user store DAO inherits from my own Abstract class, but at the moment it ONLY contains a reference to the template:

AbstractHBaseStore:

Code:

public abstract class AbstractHBaseStore {

    private HbaseTemplate hbaseTemplate;

    protected AbstractHBaseStore(HbaseTemplate template) {
        this.hbaseTemplate = template;
    }

    public HbaseTemplate getHbaseTemplate() {
        return hbaseTemplate;
    }

    public void setHbaseTemplate(HbaseTemplate hbaseTemplate) {
        this.hbaseTemplate = hbaseTemplate;
    }
}

UserStoreHBaseImpl:

Code:



public class UserStoreHBaseImpl extends AbstractHBaseStore implements UserStore, RowMapper<User> {

    public static final String TABLE_NAME = "User";
    public static final String COL_FAMILY_NAME_PROFILE = "profile";
    public static final String COL_NAME_EMAIL_ADDRESS = "email_address";
    public static final String COL_NAME_GIVEN_NAME = "given_name";
    public static final String COL_NAME_SUR_NAME = "sur_name";

    public UserStoreHBaseImpl(HbaseTemplate hbaseTemplate) {
        super(hbaseTemplate);
    }

    @Override
    public User findUserByPrimaryKey(String key) {

        User user = getHbaseTemplate().get(TABLE_NAME, key, this);
        return(user);

    }

    @Override
    public void persistUser(final String key, final User user) {

        getHbaseTemplate().execute(TABLE_NAME, new TableCallback<Object>() {
            @Override
            public Object doInTable(HTable table) throws Throwable {

                Put put = new Put(key.getBytes());
                put.add(COL_FAMILY_NAME_PROFILE.getBytes(), COL_NAME_EMAIL_ADDRESS.getBytes(), user.getEmailAddress().getBytes());
                put.add(COL_FAMILY_NAME_PROFILE.getBytes(), COL_NAME_GIVEN_NAME.getBytes(), user.getGivenName().getBytes());
                put.add(COL_FAMILY_NAME_PROFILE.getBytes(), COL_NAME_SUR_NAME.getBytes(), user.getSurName().getBytes());

                table.put(put);

                return(null);

            }
        });

    }

    @Override
    public void delete(final String key) {

        getHbaseTemplate().execute(TABLE_NAME, new TableCallback<Object>() {
            @Override
            public Object doInTable(HTable hTable) throws Throwable {

                Delete delete = new Delete(key.getBytes());
                hTable.delete(delete);

                return(null);

            }
        });

    }

    @Override
    public User mapRow(Result result, int i) throws Exception {

        String email = Bytes.toString(result.getValue(COL_FAMILY_NAME_PROFILE.getBytes(), COL_NAME_EMAIL_ADDRESS.getBytes()));
        String givenName = Bytes.toString(result.getValue(COL_FAMILY_NAME_PROFILE.getBytes(), COL_NAME_GIVEN_NAME.getBytes()));
        String surName = Bytes.toString(result.getValue(COL_FAMILY_NAME_PROFILE.getBytes(), COL_NAME_SUR_NAME.getBytes()));

        User user = new User();
        user.setEmailAddress(email);
        user.setGivenName(givenName);
        user.setSurName(surName);

        return(user);

    }
}

The problem that I am having is that the first call to my UserStoreHBaseImpl class (and therefore, the first call to hbaseTemplate) succeeds, but any other call after the first fails with the following errors:

Code:

12/07/11 14:38:12 INFO zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session
12/07/11 14:38:12 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x138779fec78000d, negotiated timeout = 40000
12/07/11 14:38:12 INFO client.HConnectionManager$HConnectionImplementation: Closed zookeeper sessionid=0x138779fec78000d
12/07/11 14:38:12 INFO zookeeper.ClientCnxn: EventThread shut down
12/07/11 14:38:12 INFO zookeeper.ZooKeeper: Session: 0x138779fec78000d closed
Inserted

org.springframework.data.hadoop.hbase.HbaseSystemException: Failed after attempts=10, exceptions:
Wed Jul 11 14:38:12 PDT 2012, org.apache.hadoop.hbase.client.HTable$5@43ce663c, java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1996e136 closed
Wed Jul 11 14:38:13 PDT 2012, org.apache.hadoop.hbase.client.HTable$5@43ce663c, java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1996e136 closed
; nested exception is org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=10, exceptions:
Wed Jul 11 14:38:12 PDT 2012, org.apache.hadoop.hbase.client.HTable$5@43ce663c, java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1996e136 closed
Wed Jul 11 14:38:13 PDT 2012, org.apache.hadoop.hbase.client.HTable$5@43ce663c, java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1996e136 closed
Wed Jul 11 14:38:14 PDT 2012, org.apache.hadoop.hbase.client.HTable$5@43ce663c, java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1996e136 closed
Wed Jul 11 14:38:15 PDT 2012, org.apache.hadoop.hbase.client.HTable$5@43ce663c, java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1996e136 closed
Wed Jul 11 14:38:17 PDT 2012, org.apache.hadoop.hbase.client.HTable$5@43ce663c, java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1996e136 closed
	at org.springframework.data.hadoop.hbase.HbaseUtils.convertHbaseException(HbaseUtils.java:42)
	at org.springframework.data.hadoop.hbase.HbaseTemplate.convertHbaseAccessException(HbaseTemplate.java:111)
	at org.springframework.data.hadoop.hbase.HbaseTemplate.execute(HbaseTemplate.java:82)
	at org.springframework.data.hadoop.hbase.HbaseTemplate.get(HbaseTemplate.java:174)
	at org.springframework.data.hadoop.hbase.HbaseTemplate.get(HbaseTemplate.java:164)
	at data.store.hbase.UserStoreHBaseImpl.findUserByPrimaryKey(UserStoreHBaseImpl.java:30)
	at data.store.hbase.UserStoreHBaseImplTest.testHBaseInsertQueryDelete(UserStoreHBaseImplTest.java:37)

It almost looks like the first call closes the connection to HBase so that subsequent calls always fail - but that doesn't make sense, unless I am improperly using the HBaseTemplate. Am I?

The JUnit test that tests this DAO:

Code:


@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations={"classpath:/spring-config/*.xml"})
public class UserStoreHBaseImplTest {

    @Autowired
    private ApplicationContext applicationContext;

    @Test
    public void testHBaseInsertQueryDelete() throws Exception {

        UserStoreHBaseImpl userStore  = (UserStoreHBaseImpl) applicationContext.getBean("userStore");



        User user = new User();
        user.setEmailAddress("test@test.com");
        user.setGivenName("Tester");
        user.setSurName("Testable");

        userStore.persistUser("test@test.com", user);
        System.out.println("Inserted");

        // retrieve it!
        User gotUser = userStore.findUserByPrimaryKey("test@test.com");
        assertNotNull(gotUser);
        assertEquals("test@test.com", gotUser.getEmailAddress());
        assertEquals("Tester", gotUser.getGivenName());
        assertEquals("Testable", gotUser.getSurName());
        System.out.println("RETRIEVED");



        // delete it!
        userStore.delete("test@test.com");
        System.out.println("deleted!");


        // try to retrieve it again
        User delUser = userStore.findUserByPrimaryKey("test@test.com");
        assertNull(delUser);



    }




}

Any help would be greatly appreciated!

Thanks.

**Costin Leau** · Jul 13th, 2012, 01:00 AM

Can you please raise a bug? It looks like the template is not properly disposing of tables.

**Costin Leau** · Jul 13th, 2012, 05:04 AM

I've pushed some fixes into master along with a nightly build. Can you please try out and let us know whether it fixes your problems or not?

Cheers,

**michaeldowling** · Jul 13th, 2012, 09:53 AM

Hi Costin,

Thanks for your reply. And your fix! Yes, this seems to fix the issue.

Did you still want me to file a bug for this issue? I'd be happy to, if it helps.

Thanks,
-Michael

**Costin Leau** · Jul 13th, 2012, 10:06 AM

No. That's fine - but in the future, go right ahead and create the issues.
Anyway, give the project a spin and let us know what you think of it.

**sujata** · Sep 6th, 2012, 12:33 AM

Originally Posted by Costin Leau

No. That's fine - but in the future, go right ahead and create the issues.
Anyway, give the project a spin and let us know what you think of it.

I am new to Hbase. I am facing the same issues as discussed here. I got the spring data library from the maven repository 1.0.0M2, please let me know where to get the fixed version of it.

Thanks in Advance
Sujata

**Costin Leau** · Sep 6th, 2012, 03:48 AM

http://www.springsource.org/spring-data/hadoop#maven
Use the latest build, meaning 1.0.0.BUILD-SNAPSHOT.

**sujata** · Sep 6th, 2012, 04:00 AM

Thank you so much Costin Leau.... I am able to proceed now

**michaeldowling** · Sep 6th, 2012, 08:30 AM

Use the following repo in your pom:

Code:

        <repository>
            <!-- Snapshots -->
            <id>spring-snapshot</id>
            <name>Spring Maven SNAPSHOT Repository</name>
            <url>http://repo.springframework.org/snapshot</url>
        </repository>

Then include the following dependency:

Code:

        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-hadoop</artifactId>
            <version>1.0.0.BUILD-SNAPSHOT</version>
        </dependency>

Hope that helps.

-Michael

**sujata** · Sep 7th, 2012, 12:20 AM

Hi Michael,

Thanks for the reply. My project is not a maven project, i had to copy the latest jar from the SNAPSHOT location.
Now i am able to proceed.

Thanks
Sujata

Thread: Possible issue with closing connections in HBaseTemplate?

Thread Tools

Display

Possible issue with closing connections in HBaseTemplate?

Tags for this Thread

Posting Permissions