OutOfMemoryError - File read & File write

[sansun2111]

Thanx a ton for your reply Dave and Anish

I happen to change the commit interval to 1 as well as to 10 still then the problem persists..
To give you more insight i am processing/parsing a EDI file(* as delimiters)
If I change my VM settings I don't get any exception as such,
but if I am not wrong as Dave said, i guess Spring Batch doesn't consume much memory, Is there something wrong with my processing logic ?

P.S : I don't want to change my VM settings as such.
Thanks ,
Sandeep.S

[sansun2111]

I happen to return the Object only when the complete File has been processed . I am retruning AF object , as specified in my code snippet. I happen to write even the Business/Processing logic in the reader itself rather than in the ItemProcessor . But If i am not wrong, that should not be the reason behind this exception.

public  Object read() throws Exception {

		
			while(itemReader.read()!=null){
                          calling some method to process  each lines
                          i am processing each line and then capturing the values from a delimited files. 
                          so beacuse of this   untill  i  completely read a file i cant  write  it.
                       
}
			result = AF;

		

		return result;

	}

680976 [main] ERROR com.abcbs.EDI837.common.MyCommandLine  - Job Terminated in error:
java.lang.OutOfMemoryError: Java heap space

[Dave Syer]

You process a whole file in ItemReader.read()? Then I guess it's up to you to control your heap usage (and you don't share the details, but it looks like your AF object could be large?).

[sansun2111]

Thanks Dave,
You mean to say Reading the whole file and then returning a object to ItemWriter is throwing OutOfMemoryException.

[Dave Syer]

I don't know, it's virtually impossible to tell from what you have shared (and hard enough in any case). Have you tried profiling? How big would you estimate the AF object to be in bytes? How big is the file it represents?

[sansun2111]

Hey Dave ,
I do agree that, its very difficult for any one as such to tell whats the cause of OutOfMemory ,

Anyways as you have asked me few things

The Size of the file i processed is around 43MB and then the the resulting output file size would defly be 15 MB after processing it , as i happen to change the VM arguements and then happen to test it , i was able 2 generate it. so i am expecting the same O/P even with out changing the VM Args.


Regarding Profiling : no I dint do any profiling as such, so i am not sure abt the Object Size .

I have got a small query,
I am processing a flat file and i want to generate a File after processing it and @ the same time i want to change the contents of the orginal file( Update the contents) @ few places ?
How can that be done ?



Thanks a lot for your time.


Thanks
Sandie.