Reading a file is a common use for Apache Camel. From using a file to kick off a larger route to simply needing to store the file content, the ability to only read a file once is important. This is easy when you have a single server with your route deployed, but what about when you deploy your route to multiple servers? Thankfully, Camel has the concept of an idempotent consumer.
A useful way to implement this is with an idempotent repository. This repository will keep track of the file being read and not allow another server to read it. It works as a lock on the file. After the file reading has been completed, there are options to either remove the row from the database (which would allow for files with the same name to be ingested later) or to keep the row there (and not allow for files of the same name to be later ingested).
No comments:
Post a Comment