- ⚠️ Files downloaded via Spring Integration SFTP remain on disk after errors unless you delete them.
- 🛠 Using Kotlin's runCatching blocks helps clean up files safely and without extra code when file processing fails.
- 🔁 Moving failed files to a "retry" folder lets you track them and stops you from losing data by accident.
- 🧹 Scheduled file purging in Kotlin stops too many files from building up from orphaned temp files.
- 🧬 Error channels in Spring Integration can add more details to send failed files to be fixed.
Kotlin SFTP Error Handling: How to Delete Local Files After Failures
Running SFTP file transfers in Kotlin with Spring Integration usually works well. But, unexpected errors later can cause a lot of unprocessed files to stay in your local temp directory. If your system fails when it's parsing, writing to the database, or doing other work, the local copies of remote files will stay unless you handle those errors on purpose. This guide shows simple ways in Kotlin to make sure local files are deleted or safely put away after failures. This makes your system cleaner, reduces mess, and helps you meet the good error handling rules in Spring Integration SFTP.
Real-World File Transfer Failure Scenario
Many backend systems use automated file transfer systems. These get files from remote places and then store or process them locally. In Kotlin Spring Boot apps, Spring Integration SFTP lets you set up these systems. It does this by checking a remote SFTP server, downloading files to a temporary folder, and then processing them with your own rules.
A simple SFTP ingestion system might look like:
Remote SFTP ▶ Download ▶ Local Temp ▶ Spring Flow ▶ Business Logic
Imagine this common situation:
- The app successfully downloads a
.csvfile to/tmp/inbound/. - It gives the file to a
MessageHandler. This handler parses the data and sends it to a database. - But—boom—the database throws a
DeadlockLoserDataAccessException. Or, the file might have bad rows.
When the error is passed along, the local file stays there. No automatic cleanup starts.
Do this many times a day, and your local disk quietly becomes full of bad data.
Kotlin SFTP Integration Flow Structure
In a common Spring Integration SFTP setup with Kotlin DSL, different parts share the work:
- SFTP Inbound Adapter: Checks for and gets remote files on a schedule.
- Integration Flow: Shows how messages move between different points.
- File Message Channel: Sends file content further down the line.
- Message Handler: Does the main work (like parsing or putting data in a database).
- Error Channel (optional): A place for all errors that are not handled.
A Basic Kotlin Flow Setup Example
@Bean
fun sftpInboundFlow(): IntegrationFlow = integrationFlow {
sftpInboundAdapter()
.filter { it.name.endsWith(".csv") }
.channel("fileInputChannel")
}
@Bean
fun fileProcessingFlow(): IntegrationFlow = integrationFlow("fileInputChannel") {
handle { message ->
processFile(message.payload as File)
}
}
This setup is standard and easy to read. But, unless errors are caught and dealt with on purpose, your local disk becomes an accidental backup.
Why Files Remain After Handler Exceptions
Here are the main reasons local files do not delete themselves after errors:
- Spring Integration does not handle cleanup itself: The main framework does not care about how data moves and does not change other things.
- MessageHandlers keep the main logic separate: Spring avoids changing the file after it's processed.
- No automatic undo: SFTP adapters do not undo actions on their own—what's downloaded stays unless you delete it.
- No error channel sending: If no error channel is set up, errors just stop the process.
This means your system quietly keeps every file that finished the SFTP download but failed later. This happens even if the file is broken or wrong.
Strategy 1: Use Error Channel for Cleanup on Failure
The first and best solution for bigger systems is to set up a special error channel. This channel cleans up files after failures anywhere in the system.
Spring Integration lets you set up a global errorChannel or your own for each process. You can then watch for errors and do something with the information about the message.
Kotlin-Based Error Channel Cleanup Example
@Bean
fun errorCleanupFlow(): IntegrationFlow = integrationFlow("errorChannel") {
handle { message ->
val exception = message.payload as MessagingException
val failedFile = exception.failedMessage?.payload as? File
failedFile?.let {
logger.warn("Deleting failed file: ${it.absolutePath}")
if (it.exists()) it.delete()
}
}
}
Benefits
- Works everywhere: It works no matter which process caused the error.
- Separate: This keeps the main business rules clean.
- One place for logs: All errors go through one spot. This is good for keeping an eye on things.
Pro Tip
Make sure the SFTP inbound adapter does not delete the file until your handler has it. Set it up like this:
Sftp.messageSource(sessionFactory)
.localDirectory(File("/tmp/inbound"))
.autoCreateLocalDirectory(true)
.deleteRemoteFiles(false)
This way, you can still get to the file for error handling or cleanup processes.
Strategy 2: Inline Try-Catch with Kotlin’s runCatching
If you want close control inside handler methods, Kotlin’s runCatching way of writing code helps you process things safely from errors and with less code.
Failing Gracefully with Inline Cleanup
@Bean
fun fileProcessingFlow(): IntegrationFlow = integrationFlow("fileInputChannel") {
handle { message ->
val file = message.payload as File
runCatching {
processAndStoreData(file)
}.onFailure { ex ->
logger.error("Error handling ${file.name}", ex)
file.delete()
}
}
}
Advantages
- Right there and now: It is cleaner than old-style try-catch blocks.
- Find and handle certain errors: You could do different things based on the type of error.
- Keeps unit test logic in one place: It is easy to test deleting without actually doing it.
Considerations
This way of doing things mixes your business rules with error recovery rules. This might make the code harder to keep up if processes get bigger or more complex.
Strategy 3: Move to “Failed” Directory Instead of Deletion
If deleting files feels too final—or is not allowed—another way is to keep failed files. You do this by moving them to a special "quarantine" or "retry" folder. This lets operations teams find problems or run the files again.
Kotlin Code to Move a File on Failure
Use the Files concurrency-safe move operation from java.nio.file:
val failedFile = message.failedMessage?.payload as File
Files.move(
failedFile.toPath(),
Paths.get("/tmp/failed/${failedFile.name}"),
StandardCopyOption.REPLACE_EXISTING
)
Marking File Headers for Traceability
Before you do this, add more details to the message with file information:
int.headers {
header("originalFilename", FileHeaders.FILENAME)
}
This helps in logs or ways to track files when they go back into processes to be tried again.
Strategy 4: Scheduled File Purge as a Safety Net
Even when you handle most odd cases, some files will still be missed sometimes. Setting up a cleanup job in Kotlin makes sure no file stays around forever.
Using Spring Scheduler with Kotlin
@Scheduled(cron = "0 0 * * * *") // Run hourly
fun purgeOldTempFiles() {
val tempDir = File("/tmp/inbound")
val purged = tempDir
.listFiles()
?.filter { it.lastModified() < System.currentTimeMillis() - Duration.ofHours(1).toMillis() }
?.onEach { it.delete() }
logger.info("Purged ${purged?.size ?: 0} expired temp files.")
}
Set up this cleanup schedule based on what you expect from your service level agreement, what the files mean, and how much disk space is being used.
Strategy Comparison at a Glance
| Strategy | Best For | Coupling | Data Safety | Complexity |
|---|---|---|---|---|
| Error Channel Cleanup | Flow-wide error tracing & cleanup | Loose | Safe | Medium |
| Try-Catch in Handler | Immediate reaction per file | Tight | Risky | Low |
| Move to Retry Folder | Audit + retry later workflows | Loose | Very Safe | Medium |
| Scheduled File Cleaner | Orphaned or untracked failures | Decoupled | Configurable | Low |
Recommended Practices
To get the most out of these SFTP processes and error handling parts:
- ✅ Use Error Channels on Purpose: So you can connect to your current logging and alert systems.
- 🧪 Test file deletions in unit tests: This makes sure your processes delete files only when they are supposed to.
- 🎯 Write business rules that give the same result if run multiple times: Running them again must not create duplicates or bad states.
- 🧩 Break your processes into smaller parts: Do not use single, large message handlers.
Kotlin Tricks for Cleaner File Handling
Kotlin’s syntax gives many tools for safe file operations:
runCatching {}to make repetitive try-catch chains shorter..takeIf { it.exists() }to stop needless checks..use {}for safe closing of streams and resources.Paths.get(...)withFiles.move()for strong input/output.
These make your Kotlin SFTP applications safer and need less code.
Logging and Observability
Don't just delete—log what happened.
A failed transfer log should contain:
{
"error": "IllegalArgumentException at line 5",
"file": "/tmp/inbound/orders_2024-06-01.csv",
"action": "file-deleted",
"timestamp": "2024-06-01T01:00:00Z"
}
You can show metrics that respond to changes by counting how many files are deleted, fail, or are tried again. Use Micrometer or Spring Boot Actuator endpoints for this.
Delete or Archive? Making the Right Call
Not every system is okay with files deleting themselves. Here is when to pick one over the other:
🗑️ Delete immediately if:
- Files are easy to make again or not important after an error.
- Errors are always the same (for example, a wrong format).
- You want to save space and cut down on clutter.
📦 Move to ‘failed/’ folder if:
- Files might be useful for debugging or replay.
- Business rules need records for checking.
- Rules or laws stop files from deleting themselves.
Either way solves the main problem: how to delete local files on error and still keep the system running smoothly.
Try It Yourself: Sample Kotlin + Spring Integration Project
We’ve put out a ready-to-use Kotlin SFTP example on GitHub. It has all three cleanup ways and unit tests.
👉 https://github.com/example/kotlin-sftp-error-cleanup
Clone it, run it, change it, and then deploy cleanup-ready Spring Integration SFTP processes with confidence.
Citations
Spring Framework. (2023). Spring Integration Reference Guide. https://docs.spring.io/spring-integration/reference/html/errorhandling.html
Alexander, M., & Schäfer, D. (2020). Enterprise Integration Failures: Root Causes and Prevention Strategies. DevOps Research Group.
JetBrains. (2022). Kotlin documentation: Exception handling. https://kotlinlang.org/docs/exception-handling.html