Problem with JpaPagingItemReader and pageSize

[jprio]

Well, what I get in the writer is an object attached to an entitymanager (because it is read by the em). So if I modify this object, the modification should be synchronized in the db on a flush, right (and it is for the first pages) ?

[traduz]

Yes you are right, but don't you agree the writer should have the responsability to update and save those changes?
So flush it in the writer.

[jprio]

Humm may be. I'm not sure. But, anyway, to do this, I have to have access to the em instanciated by Spring (from the emf) for this transaction (i.e. this page) and I don't know how I could do this (AFAIK there is no way to get the "current em" with JPA).

[traduz]

Use JpaItemWriter.

[jprio]

Yep, thanx. I'll try tomorrow (processor + JpaItemWriter) and I'll let you know.

[jprio]

Couldn't wait tomorrow
I have my JpaItemReader (pageSize = 8), 15 persons in my db. I have a processor which adds 2 addresses per person. I use a JpaItemWriter and I have a commit interval = 8. At the end of my test, I have ... 46 adresses (???) in my db.
If I change the commit interval to 4 and pageSize to 4, I have 54 adresses in my db.
I works only if pageSize > total number of record in the db.

[traduz]

Do you mind posting the code and config you used for JpaItemWriter please?
There's a Jira related to it, why flush/clear was moved, damn old: https://jira.springsource.org/browse/BATCH-1110
And are you sure about those numbers? Are you clearing the address before testing? So we'll know how many updates we had.

[jprio]

The relevant parts :

<batch:job id="minimal" job-repository="jobRepository"
restartable="true">

<batch:step id="step1">
<batch:tasklet>
<batch:chunk reader="jpaPersonReader" processor="addAdressToPersonProcessor"
writer="jpaPersonWriter" commit-interval="4" />
</batch:tasklet>
</batch:step>
</batch:job>

<bean id="addAdressToPersonProcessor" class="org.jp.spring_batch_labs.AddAddressToPerson Processor" />

<bean id="jpaPersonReader"
class="org.springframework.batch.item.database.Jpa PagingItemReader">
<property name="entityManagerFactory" ref="entityManagerFactory" />
<property name="queryString" value="select p from Person p" />
<property name="pageSize" value="4" />
</bean>

<bean id="jpaPersonWriter" class="org.springframework.batch.item.database.Jpa ItemWriter">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>

The processor :

public class AddAddressToPersonProcessor implements ItemProcessor<Person, Person> {
private static final Logger LOG = Logger
.getLogger(AddAddressToPersonProcessor.class);

public Person process(Person person) throws Exception {
LOG.info("processing : " + person);
Address address = new Address();
address.setCp("92160");
address.setStreet("street1");
person.addAddress(address);

Address address2 = new Address();
address2.setCp("92160");
address2.setStreet("street2");
person.addAddress(address2);
return person;
}
}


My u.t. :

public class SimpleTestJpa {

private static final Logger LOG = Logger.getLogger(SimpleTestJpa.class);
static ApplicationContext ac = new ClassPathXmlApplicationContext(
new String[] { "spring-test-jpa.xml" });

@BeforeClass
public static void beforeClass() {
// Création du schéma pour spring batch
SimpleJdbcTemplate jdbcTemplate = (SimpleJdbcTemplate) ac
.getBean("jdbcTemplate");
Resource resource = new ClassPathResource(
"/create_drop_spring_batch.sql");
SimpleJdbcTestUtils.executeSqlScript(jdbcTemplate, resource, true);

// dbunit : population des tables
DataSource dataSource = (DataSource) ac.getBean("dataSource");
Connection con = DataSourceUtils.getConnection(dataSource);
IDatabaseConnection dbUnitCon = new DatabaseConnection(con);

try {
IDataSet dataSet = new FlatXmlDataSet(new FileInputStream(
"./src/test/resources/persons_dbunit.xml"));
DatabaseOperation.REFRESH.execute(dbUnitCon, dataSet);
} catch (Exception e) {
e.printStackTrace();
} finally {
DataSourceUtils.releaseConnection(con, dataSource);
}
}

@Test
public void simpleTest() throws Exception {
Job job = (Job) ac.getBean("minimal");
JobLauncher jobLauncher = (JobLauncher) ac.getBean("jobLauncher");
// Lancement effectif
JobExecution je = jobLauncher.run(job, new JobParametersBuilder()
.addString("param1", "1").toJobParameters());
}

@AfterClass
public static void afterClass() {
DataSource dataSource = (DataSource) ac.getBean("dataSource");
int count = 0;
try {
ResultSet rs = dataSource.getConnection().createStatement()
.executeQuery("select count(*) from Address");
rs.next();
count = rs.getInt(1);
rs.close();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
LOG.info("Nbre d'enregistrements : " + count);

}
}


My dataset :
<?xml version='1.0' encoding='UTF-8'?>
<dataset>
<PERSON ID='1' NAME="name1"/>
<PERSON ID='2' NAME="name2"/>
<PERSON ID='3' NAME="name3"/>
<PERSON ID='4' NAME="name3"/>
<PERSON ID='5' NAME="name3"/>
<PERSON ID='6' NAME="name3"/>
<PERSON ID='7' NAME="name3"/>
<PERSON ID='8' NAME="name3"/>
<PERSON ID='9' NAME="name3"/>
<PERSON ID='10' NAME="name3"/>
<PERSON ID='11' NAME="name3"/>
<PERSON ID='12' NAME="name3"/>
<PERSON ID='13' NAME="name3"/>
<PERSON ID='14' NAME="name3"/>
<PERSON ID='15' NAME="name3"/>
</dataset>

Person :
@Entity
public class Person {
@Id
@GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
private String name;
@OneToMany(cascade = CascadeType.ALL)
@JoinTable(name = "PERSON_ADDRESS", joinColumns = { @JoinColumn(name = "PERSON_ID") }, inverseJoinColumns = { @JoinColumn(name = "ADDRESS_ID") })
private Set<Address> addresses = new HashSet<Address>();
...+get/set


Address :
@Entity
public class Address {
@Id
@GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
private String cp;
private String street;
...+get/set

[traduz]

Thanks, next time use the code tag so will be better to read it my friend. I'll take a look at it.
Also you didn't answered, are you clearing the address before testing? So we'll know how many updates we had.

[jprio]

Sorry about the 'code tag'
I forgot to answer for the address : yes. The db is empty and just filled with the dataset above (only 'persons' wo address).