Any way to get the total count of input data set

[wordywordy]

My use case is to enable my step able to report the percentage of completion on each commit. To achieve that, I need count the total size of input data set (e.g. the no. of line in a file).

To my understanding, it has no out-of-the-box way to do the titled thing in SB. Please correct me if don't.

If I want to implement such thing what, I think think of some of the following plug-in point to calculate the total item count:

- at StepExecution.beforeStep
- at Reader

which one is better or any comments on these? Or any new direction for doing this?

[lucasward]

It's a bit of a tough problem. The only way that wouldn't require you to read through the whole file once to determine a line count would be to use the size in bytes. If you have fixed length input, it would be easy, since you know exactly how many bytes are in a line, multiplying the current line count by that number and dividing by the total size would give you the percentage. I have no good ideas on how to do the same with delimited input though. I suppose we could use the number of bytes read instead of lines read in the reader, which could help with that.

[wordywordy]

Thanks for your reply.

I think we can divide the problem into where and how. I think you have suggested some ways for 'how'. How about where?

- at StepExecution.beforeStep?
- at Reader?