off_broadway_files

An Elixir Broadway producer that reads files from a directory.

It periodically looks for new files in a specificed directory, passes them to the Broadway pipeline, then moves them to an archive directory on success, or a failure directory failure.

It parses the date from the file name to determine where to archive them. Files look like traffic-20260108-2049.log.

Installation

If available in Hex, the package can be installed by adding off_broadway_files to your list of dependencies in mix.exs:

def deps do
  [
    {:off_broadway_files, "~> 0.7.0"}
  ]
end

Documentation can be found at https://hexdocs.pm/off_broadway_files.