{"id":381,"date":"2025-08-13T13:07:12","date_gmt":"2025-08-13T12:07:12","guid":{"rendered":"https:\/\/davagordon.co.uk\/blog\/?p=381"},"modified":"2025-08-13T13:08:22","modified_gmt":"2025-08-13T12:08:22","slug":"s3-images-webp-conversion","status":"publish","type":"post","link":"https:\/\/davagordon.co.uk\/blog\/s3-images-webp-conversion\/","title":{"rendered":"S3 Images WebP Conversion: Batch Convert with Node.js Script"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">If you\u2019re hosting lots of S3 images, converting them to WebP can dramatically reduce file size and improve load times. This S3 images WebP conversion guide uses a Node.js script to automate the process.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But converting hundreds or thousands of images in S3 manually? No thanks. That\u2019s why I built a Node.js script that <strong>automatically converts your S3 images to WebP<\/strong> and uploads them back to your bucket.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here\u2019s how it works and how you can use it.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Why S3 Images WebP Conversion with This Script Is Useful<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Batch conversion:<\/strong> Handles all your images in one go.<\/li>\n\n\n\n<li><strong>Smart concurrency:<\/strong> Converts multiple images in parallel without overloading S3.<\/li>\n\n\n\n<li><strong>Selective processing:<\/strong> Skip folders or only process specific prefixes.<\/li>\n\n\n\n<li><strong>Dry-run mode:<\/strong> See what would happen without actually uploading files.<\/li>\n\n\n\n<li><strong>Logging:<\/strong> Keeps track of progress and writes failed keys to <code>failed-keys.txt<\/code>.<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1846\" height=\"664\" src=\"https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/s3progress.png\" alt=\"\" class=\"wp-image-383\" srcset=\"https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/s3progress.png 1846w, https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/s3progress-300x108.png 300w\" sizes=\"auto, (max-width: 1846px) 100vw, 1846px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">How S3 Images WebP Conversion Works<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The script works in a few simple steps:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>List objects<\/strong> in your S3 bucket.<\/li>\n\n\n\n<li><strong>Skip files<\/strong> that are already in WebP format.<\/li>\n\n\n\n<li>Apply <strong>include\/exclude rules<\/strong> based on prefixes.<\/li>\n\n\n\n<li>Convert images using <a class=\"\" href=\"https:\/\/www.npmjs.com\/package\/sharp\"><code>sharp<\/code><\/a>.<\/li>\n\n\n\n<li>Upload the <code>.webp<\/code> images back to your S3 bucket.<\/li>\n<\/ol>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1536\" height=\"1024\" src=\"https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/s3webpdiagram.png\" alt=\"\" class=\"wp-image-385\" srcset=\"https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/s3webpdiagram.png 1536w, https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/s3webpdiagram-300x200.png 300w\" sizes=\"auto, (max-width: 1536px) 100vw, 1536px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Getting Started<\/h2>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Clone the repository:<\/strong><\/li>\n<\/ol>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\ngit clone https:\/\/github.com\/DavaGordon\/aws-s3-webp-images.git\ncd aws-s3-webp-images\n<\/pre><\/div>\n\n\n<ol start=\"2\" class=\"wp-block-list\">\n<li><strong>Install dependencies:<\/strong><\/li>\n<\/ol>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\nnpm install\n<\/pre><\/div>\n\n\n<ol start=\"3\" class=\"wp-block-list\">\n<li><strong>Configure your bucket and region<\/strong> in the script:<\/li>\n<\/ol>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: jscript; title: ; notranslate\" title=\"\">\nconst S3 = new AWS.S3({ region: &#039;YOUR_S3_BUCKET_REGION&#039; });\nconst bucket = &#039;YOUR_BUCKET_NAME&#039;;\n<\/pre><\/div>\n\n\n<h2 class=\"wp-block-heading\">Running the Script<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Basic run:<\/strong><\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\nnode convert-existing-batch.js\n<\/pre><\/div>\n\n\n<p class=\"wp-block-paragraph\"><strong>Command-line options:<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Option<\/th><th>Type<\/th><th>Description<\/th><\/tr><\/thead><tbody><tr><td><code>--exclude \"prefix1,prefix2\"<\/code><\/td><td>string<\/td><td>Skip files in specified prefixes<\/td><\/tr><tr><td><code>--include \"prefix\"<\/code><\/td><td>string<\/td><td>Only process keys starting with this prefix<\/td><\/tr><tr><td><code>--dry-run<\/code><\/td><td>boolean<\/td><td>Logs actions without uploading converted files<\/td><\/tr><tr><td><code>--concurrency N<\/code><\/td><td>number<\/td><td>Max parallel conversions (default: 5)<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Examples:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Convert only images in the <code>products\/<\/code> folder:<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\nnode convert-existing-batch.js --include &quot;products\/&quot;\n<\/pre><\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>Skip the <code>backup\/<\/code> folder:<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\nnode convert-existing-batch.js --exclude &quot;backup\/&quot;\n<\/pre><\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>Test conversion without uploading:<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\nnode convert-existing-batch.js --dry-run\n<\/pre><\/div>\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"2034\" height=\"646\" src=\"https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/s3dryrun.png\" alt=\"S3 images WebP conversion script running in Node.js\" class=\"wp-image-382\" srcset=\"https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/s3dryrun.png 2034w, https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/s3dryrun-300x95.png 300w\" sizes=\"auto, (max-width: 2034px) 100vw, 2034px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Logging &amp; Progress During S3 Images WebP Conversion<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The script prints progress every 50 images or at the end.<\/li>\n\n\n\n<li>Skipped files and successful conversions are logged.<\/li>\n\n\n\n<li>Failed conversions are written to <code>failed-keys.txt<\/code>.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Tips &amp; Notes<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Adjust concurrency<\/strong> to balance speed and resource usage.<\/li>\n\n\n\n<li>Existing <code>.webp<\/code> files are skipped automatically.<\/li>\n\n\n\n<li>Make sure your IAM role or <a href=\"https:\/\/davagordon.co.uk\/blog\/tag\/devops\/\" title=\"DevOps\">AWS<\/a> credentials allow <code>s3:GetObject<\/code>, <code>s3:PutObject<\/code>, and <code>s3:ListBucket<\/code>.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Troubleshooting:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Missing credentials error:<\/strong> Ensure your EC2 instance has the correct IAM role or AWS credentials.<\/li>\n\n\n\n<li><strong>Sharp installation issues:<\/strong> On Linux, you may need build tools:<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\nsudo apt install -y build-essential\n<\/pre><\/div>\n\n\n<h2 class=\"wp-block-heading\">Try It Out<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Ready to speed up your S3 images? Check out the script and full instructions on GitHub:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\ud83d\udc49 <a href=\"https:\/\/github.com\/DavaGordon\/aws-s3-webp-images\" title=\"\">S3 WebP Converter <\/a><a href=\"https:\/\/github.com\/DavaGordon\/aws-s3-webp-images\" target=\"_blank\" rel=\"noopener\" title=\"\">Repository<\/a><\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"596\" src=\"https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/clicktoclone-1024x596.png\" alt=\"\" class=\"wp-image-392\" srcset=\"https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/clicktoclone-1024x596.png 1024w, https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/clicktoclone-300x175.png 300w, https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/clicktoclone-768x447.png 768w, https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/clicktoclone-1536x894.png 1536w, https:\/\/davagordon.co.uk\/blog\/wp-content\/uploads\/2025\/08\/clicktoclone.png 1838w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This script is a simple way to improve load times, save bandwidth, and optimise image-heavy websites.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you\u2019re hosting lots of S3 images, converting them to WebP can dramatically reduce file size and improve load times. This S3 images WebP conversion guide uses a Node.js script to automate the process. But converting hundreds or thousands of images in S3 manually? No thanks. That\u2019s why I built a Node.js script that automatically [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":394,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-381","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorised"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/davagordon.co.uk\/blog\/wp-json\/wp\/v2\/posts\/381","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/davagordon.co.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/davagordon.co.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/davagordon.co.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/davagordon.co.uk\/blog\/wp-json\/wp\/v2\/comments?post=381"}],"version-history":[{"count":6,"href":"https:\/\/davagordon.co.uk\/blog\/wp-json\/wp\/v2\/posts\/381\/revisions"}],"predecessor-version":[{"id":397,"href":"https:\/\/davagordon.co.uk\/blog\/wp-json\/wp\/v2\/posts\/381\/revisions\/397"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/davagordon.co.uk\/blog\/wp-json\/wp\/v2\/media\/394"}],"wp:attachment":[{"href":"https:\/\/davagordon.co.uk\/blog\/wp-json\/wp\/v2\/media?parent=381"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/davagordon.co.uk\/blog\/wp-json\/wp\/v2\/categories?post=381"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/davagordon.co.uk\/blog\/wp-json\/wp\/v2\/tags?post=381"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}