Alpakka S3: Retrying internal errors


(Michael Pisula) #1

Hi there,
we are currently facing intermitting internal errors from S3 in our Alpakka application. Error message is “We encountered an internal error. Please try again.”
Our application reads a large-ish number of files from S3 (usually 5000-10000 in a batch) before processing them. From what I read on the AWS forums there internal errors can occur, and it is not unheard of to get 1 for 5000 requests. The AWS S3 best practices suggest retrying in case of these internal errors.
I was wondering if it would make sense to do the retry directly in the Alpakka S3 code instead of the application code, as it would be far easier to retry the single failed request, and not the complete stream.
Any opinions?
Cheers,
Michael


(Enno) #2

Yes, that sounds reasonable. Would you be able to suggest the required changes in a Pull Request?


(Michael Pisula) #3

Sure thing :-)