FormatterLineAggregator, BeanWrapperFieldExtractor, ItemProcessor

[Trunks]

I have a basic question on how to use the 3 clases, FieldExtractor, Aggregator, Processor.

I have a fixed width file I need to generate that has multiple record types some of which can have up to as many as 100 fields (probably more got sick of counting).

I think I understand the general relationship, I would use a reader (from db) to create my domain objects. The Extractor to reflectively create my fieldset for writing to the file. The ItemProcessor would sit in the middle and would be responsible for transforming any attributes that arent simply 1:1 with my db input source. Then the aggregator would take the fieldset and push out to the file in the fixed width format.

The question is how do I delineate the lengths of the strings/digits for the 100+ fields? One colossal printf format property string, whose tokens have to match the order of an object array of 100 elements? Seems like a maintenance nightmare. I am guessing I must be missing something here.


I am using the following as my reference:

	<bean id="tradeLineAggregator"
		class="org.springframework.batch.item.file.transform.FormatterLineAggregator">
		<property name="fieldExtractor">
			<bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
				<property name="names" value="isin,quantity,price,customer" />
			</bean>
		</property>
		<property name="format" value="TRAD%-12s%-3d%6s%-9s" />
	</bean>

Thanks in advance
-Trunks

[DHGarrette]

Unfortunately, you have it exactly right: the current way is to do it is to use a big "format". Perhaps you could create some kind of FormatFactoryBean that turns a list of fields into a format String.

[hailspring]

I had similar issue in one of my other threads. It used to be pretty straight-forward in 1.x where we had to just specify the ranges like: 1-2,4,5-9, 10 etc.

[DHGarrette]

I came up with a quick pair of implementations to make this process easier:

1. LineFormatFactoryBean
Using a FactoryBean, you can split up the format into individual parts to make it clearer:

<bean id="lineAggregator" class="...FormatterLineAggregator">
    <property name="fieldExtractor">
        <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
            <property name="names" value="name,credit" />
        </bean>
    </property>
    <property name="format">
        <bean class="org.springframework.batch.item.file.transform.LineFormatFactoryBean">
            <property name="fields">
                <list>
                    <value>%-9s</value>   <!-- name -->
                    <value>%-2.0f</value> <!-- credit -->
                </list>
            </property>
        </bean>
    </property>
</bean>

public class LineFormatFactoryBean implements FactoryBean {

    private String[] fields;

    public Object getObject() throws Exception {
        StringBuilder sb = new StringBuilder();
        for (String field : fields) {
            sb.append(field);
        }
        return sb.toString();
    }

    public Class<String> getObjectType() {
        return String.class;
    }

    public boolean isSingleton() {
        return false;
    }

    public void setFields(String[] fields) {
        this.fields = fields;
    }
}

2. BeanWrapperLineAggregator
If you're using a BeanWrapperLineExtractor, then you can simplify the configuration of the LineAggregator by just passing in a map of field names and their individual formats. The BeanWrapperLineAggregator will extract the properties and format them in the specified order:

<bean id="lineAggregator" class="...BeanWrapperLineAggregator">
    <property name="fields">
        <map>
            <entry key="name"   value="%-9s"/>
            <entry key="credit" value="%-2.0f"/>
        </map>
    </property>
</bean>

public class BeanWrapperLineAggregator<T> implements LineAggregator<T>, InitializingBean {

    private FormatterLineAggregator<T> formatterLineAggregator = new FormatterLineAggregator<T>();
    private Map<String, String> fields;

    public void afterPropertiesSet() throws Exception {
        String[] fieldNames = new String[fields.size()];
        String[] formats = new String[fields.size()];
        int i = 0;
        for (Entry<String, String> field : fields.entrySet()) {
            fieldNames[i] = field.getKey();
            formats[i] = field.getValue();
            i++;
        }

        BeanWrapperFieldExtractor<T> fieldExtractor = new BeanWrapperFieldExtractor<T>();
        fieldExtractor.setNames(fieldNames);
        formatterLineAggregator.setFieldExtractor(fieldExtractor);

        LineFormatFactoryBean lineFormat = new LineFormatFactoryBean();
        lineFormat.setFields(formats);
        formatterLineAggregator.setFormat((String) lineFormat.getObject());
    }

    public String aggregate(T item) {
        return this.formatterLineAggregator.aggregate(item);
    }

    /**
     * @param fields A map of fields where the name of the field is the key and
     *        the format is the value
     */
    public void setFields(Map<String, String> fields) {
        this.fields = fields;
    }
}

[Trunks]

Thanks for the response DHGarrette. It is very likely I will end up implementing something similar to your approach.

I have been building out a proof of concept for the multiple record type scenario, and its proving to be a little more difficult than I anticipated.

I used the multilineOrder job as my sample and got it working, but it leaves a few questions unanswered...

1.) No where in there do they implement the FieldExtractor interface. Though I suppose that static class in the processor serves the same purpose? Is that common practice or should I abstract that code out and do it by implementing the FieldExtractor?

2.) I couldn't figure out for the life of me how they write out the file header and file trailer record. I had to do it implementing the Header/FooterCallback, but I do not see anything like that in the source. Maybe I just need to setup a project and test it out, but I am pretty sure multilineorderjob does not handle the file header or file footer properly.

3.) Is the multilineOrderJob the preferred way of handling multiple record types? I was looking at a post from Jean Swart
(http://forum.springsource.org/archiv...p/t-68626.html)
using the ClassifierCompositeItemWriter class, but I was unable to get that working, and I can't seem to find any documentation on it. I am little concerned because it seems were forcing the processor to do more than just data transforms prior to writing, its turning almost into the catch all prior to writing.

Thanks again in advance.

[DHGarrette]

I agree with your comments. After a brief stroll through the change log, it appears that this sample is quite out of date. It has not been updated to reflect the classes that we've added in the last 8 months and, therefore, the way it is working is not the best way. I'll will overhaul it tonight and get it into a state where it can be properly used as an sample.