Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Any support in Spring Batch for reading multiple files under zip

I am looking for loading people records from multiple files based on location. Is there any easy support by Spring batch to load multiple files named location weise?
Easy Country_people.zip
-> Location1 (folder1) containing 3 text files(people_education.txt, people_address.txt, people_income.txt)

 -> Location2 (folder2) containing 3 text files(people_education.txt, people_address.txt, people_income.txt)

 -> Location3 (folder3) containing 3 text files(people_education.txt, people_address.txt, people_income.txt)

>Solution :

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

You can try using Partitioner to get the data from multiple files

https://docs.spring.io/spring-batch/docs/current/reference/html/scalability.html#partitioning

Or https://docs.spring.io/spring-framework/docs/4.0.0.RELEASE/javadoc-api/org/springframework/core/io/support/PathMatchingResourcePatternResolver.html

https://docs.spring.io/spring-batch/docs/current/api/org/springframework/batch/item/file/MultiResourceItemReader.html

public class CustomMultiResourcePartitioner implements Partitioner {

@Override
public Map<String, ExecutionContext> partition(int gridSize) {
    Map<String, ExecutionContext> map = new HashMap<>(gridSize);
    int i = 0, k = 1;
    for (Resource resource : resources) {
        ExecutionContext context = new ExecutionContext();
        Assert.state(resource.exists(), "Resource does not exist: " 
          + resource);
        context.putString(keyName, resource.getFilename());
        context.putString("opFileName", "output"+k+++".xml");
        map.put(PARTITION_KEY + i, context);
        i++;
    }
    return map;
}

}

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading