S3 Images WebP Conversion: Batch Convert with Node.js Script

• Uncategorised
• 0 Comments

If you’re hosting lots of S3 images, converting them to WebP can dramatically reduce file size and improve load times. This S3 images WebP conversion guide uses a Node.js script to automate the process.

But converting hundreds or thousands of images in S3 manually? No thanks. That’s why I built a Node.js script that automatically converts your S3 images to WebP and uploads them back to your bucket.

Here’s how it works and how you can use it.

Why S3 Images WebP Conversion with This Script Is Useful

• Batch conversion: Handles all your images in one go.
• Smart concurrency: Converts multiple images in parallel without overloading S3.
• Selective processing: Skip folders or only process specific prefixes.
• Dry-run mode: See what would happen without actually uploading files.
• Logging: Keeps track of progress and writes failed keys to failed-keys.txt.

How S3 Images WebP Conversion Works

The script works in a few simple steps:

1. List objects in your S3 bucket.
2. Skip files that are already in WebP format.
3. Apply include/exclude rules based on prefixes.
4. Convert images using sharp.
5. Upload the .webp images back to your S3 bucket.

Getting Started

1. Clone the repository:

git clone https://github.com/DavaGordon/aws-s3-webp-images.git
cd aws-s3-webp-images

2. Install dependencies:

npm install

3. Configure your bucket and region in the script:

const S3 = new AWS.S3({ region: 'YOUR_S3_BUCKET_REGION' });
const bucket = 'YOUR_BUCKET_NAME';

Running the Script

Basic run:

node convert-existing-batch.js

Command-line options:

Examples:

• Convert only images in the products/ folder:

node convert-existing-batch.js --include "products/"

• Skip the backup/ folder:

node convert-existing-batch.js --exclude "backup/"

• Test conversion without uploading:

node convert-existing-batch.js --dry-run

Logging & Progress During S3 Images WebP Conversion

• The script prints progress every 50 images or at the end.
• Skipped files and successful conversions are logged.
• Failed conversions are written to failed-keys.txt.

Tips & Notes

• Adjust concurrency to balance speed and resource usage.
• Existing .webp files are skipped automatically.
• Make sure your IAM role or AWS credentials allow s3:GetObject, s3:PutObject, and s3:ListBucket.

Troubleshooting:

• Missing credentials error: Ensure your EC2 instance has the correct IAM role or AWS credentials.
• Sharp installation issues: On Linux, you may need build tools:

sudo apt install -y build-essential

Try It Out

Ready to speed up your S3 images? Check out the script and full instructions on GitHub:

👉 S3 WebP Converter Repository

This script is a simple way to improve load times, save bandwidth, and optimise image-heavy websites.