In a previous post, I had explored uploading files to S3 using putObject and its limitations. As recommended by AWS for any files larger than 100MB we should use multipart upload. There are a couple of ways to achieve this. I’ll start with the simplest approach. S3 multipart upload As the name suggests we can use the SDK to upload our object in parts instead of one big request. The AWS APIs require a lot of redundant information to be sent with every request, so I wrote a small abstraction layer.

I recently had to work with S3 & AWS Java SDK for a few different file operation scenarios. I was aware of a few different approaces and learnt a few more. Surprisingly there were actually more than one right ways of using the SDK. I’ll be talking about the AWS SDK for Java 2 using Java 11. S3 localstack configurations Localstack is the best tool for working with AWS locally. It provides a local test framework for developing against AWS.

I have predominantly been a Java developer for over 10 years and worked with various message queuing technologies in recent years. I had the opportunity to get exposure to Kafka as one of our clients wanted to use Kafka in their Proof of Concept. While I had heard about Kafka for a long time, I never really understood its features and depth. Once I actually understood its power, I immediately saw how well it solved a host of challenges I had faced in my previous work where I had to work around them.