bulk operations with spring roo..

[Borius]

I want to add List<MyEntities> in one request (not in thousand)

In hibernate i can make this like here http://docs.jboss.org/hibernate/core...#batch-inserts

But i don't know how to do same in roo...

[wallenborn]

In JPA (which is used by roo) this looks like

@PersistenceContext
private EntityManager entityManager;

@Transactional
public void storeList(List<MyEntity> entities) {
  int imported = 0;
  for (MyEntity e: entities) {
    entityManager.persist(e);
    if (++imported % 50 == 0) {
      entityManager.flush();
      entityManager.clear();
    }
  }
}

I think you can replace entityManager.persist(e) with the rooish e.persist() (which is @Transactional, too), but you don't have to. The @Transactional annotation requires a transaction manager to handle the transaction, which is convenient if you have many such methods. But if you want to handle the transaction yourself just like in Hibernate, you can do this, too. This looks more familiar to Hibernate developers, but can become more cumbersome in the long run:

@PersistenceUnit
private EntityManagerFactory entityManagerFactory;

public void storeList(List<MyEntity> entities) {
  EntityManager entityManager = entityManagerFactory.createEntityManager();
  entityManager.getTransaction().begin();
  int imported = 0;
  for (MyEntity e: entities) {
    entityManager.persist(e);
    if (++imported % 50 == 0) {
      entityManager.flush();
      entityManager.clear();
    }
  }
  entityManager.getTransaction().commit();
  entityManager.close();
}

[Borius]

Thanks! It works and rock!=)
I was searching for this solution:

In JPA (which is used by roo) this looks like

@PersistenceContext
private EntityManager entityManager;

@Transactional
public void storeList(List<MyEntity> entities) {
  int imported = 0;
  for (MyEntity e: entities) {
    entityManager.persist(e);
    if (++imported % 50 == 0) {
      entityManager.flush();
      entityManager.clear();
    }
  }
}

But i have one more question..

I've made some benchmark (with bulk insert) and get strange result.
For example. When I add first time 10k entities it take about 4sec, and next time (and other) ~1.5sec.
When i add without bulk (just calling persist for all entities) it take every time same time (for 1k entities ~10sec).
I don't understand why?

[Andrew Swan]

@PersistenceContext private EntityManager entityManager;
@Transactional public void storeList(List<MyEntity> entities) {
    int imported = 0;
    for (MyEntity e: entities) {
        entityManager.persist(e);
        if (++imported % 50 == 0) {
            entityManager.flush();
            entityManager.clear();
        }
    }
 }

Indeed with Roo's use of the Active Record pattern, this can be abbreviated to:

@Transactional
public void storeList(List<MyEntity> entities) {
    int imported = 0;
    for (MyEntity e : entities) {
        e.persist();  // 1. Roo introduces this method via an ITD
         if (++imported % 50 == 0) {
            e.flush();  // 2. ... and this one
            e.clear();  // 3. ... and this one
        }
    }
 }

[Andrew Swan]

Actually for bulk operations, you might want to investigate the use of Spring Batch (log a JIRA ticket if you would like to see Roo support this, and we'll see if there's wider interest in such a feature).

[Borius]

[CODE]@PersistenceContext private EntityManager entityManager;

@Transactional
public void storeList(List<MyEntity> entities) {
    int imported = 0;
    for (MyEntity e : entities) {
        e.persist();  // 1. Roo introduces this method via an ITD
         if (++imported % 50 == 0) {
            e.flush();  // 2. ... and this one
            e.clear();  // 3. ... and this one
        }
    }
 }

Thanks, it works (this is best solution)

Is there any options for help in improvement spring roo?
I realy want to help in developing this great project=)

[Andrew Swan]

Is there any options for help in improvement spring roo?
I realy want to help in developing this great project=)

Great! Please see the community page for some ideas. Even just helping to answer forum questions would free up more of the core developers' time.

[sudhakarmuthu]

I have similar scenario.
Borius, did you add the "storeList(List<MyEntity> entities)" method in **_Roo_Entity.aj file.

I was reading that it is not good option to edit **Roo_Entity.aj file. if then what is alternate class to add the method.

[osroca]

Edit *_Roo_*.aj files is not a good idea. Roo is the owner of these files and it overwrites AJs when it detects a change in .java file.

You must to declare your own methods directly in the .java class, even if you want to customize one of the methods generated by Roo in AJs you must to pushin the method to the .java file and make your changes there.