aws|March 31, 2020|1 min read

How to upload files on AWS S3 by using curl, without having command line aws or other tool

TL;DR

You can upload to S3 with just curl by constructing the Authorization header with your access key and a signed string. Useful on machines without the AWS CLI installed.

How to upload files on AWS S3 by using curl, without having command line aws or other tool

Introduction

There were few files that I need to take backup from a machine that I recently launched. The machine neither had aws command line utility, nor any other code by which I could upload my files on aws s3.

Curl the savior

I already wrote few useful commands for curl

By using curl, you can actually upload the file on aws s3. The requirement is that you must have the access key and the secret key. Lets write a shell script.


# about the file
file_to_upload=<file you want to upload>
bucket=<your s3 bucket name>
filepath="/${bucket}/${file_to_upload}"

# metadata
contentType="application/x-compressed-tar"
dateValue=`date -R`
signature_string="PUT\n\n${contentType}\n${dateValue}\n${filepath}"

#s3 keys
s3_access_key=<your s3 access key>
s3_secret_key=<your s3 secret key>

#prepare signature hash to be sent in Authorization header
signature_hash=`echo -en ${signature_string} | openssl sha1 -hmac ${s3_secret_key} -binary | base64`

# actual curl command to do PUT operation on s3
curl -X PUT -T "${file_to_upload}" \
  -H "Host: ${bucket}.s3.amazonaws.com" \
  -H "Date: ${dateValue}" \
  -H "Content-Type: ${contentType}" \
  -H "Authorization: AWS ${s3_access_key}:${signature_hash}" \
  https://${bucket}.s3.amazonaws.com/${file_to_upload}

I tested this script on CentOS-7 and CentOS-8.

Related Posts

Static Website Hosting with AWS S3 and Cloudflare

Static Website Hosting with AWS S3 and Cloudflare

Static websites have several advantages over dyanamic websites. If you are…

Troubleshoot AWS Lambda unknown error!

Troubleshoot AWS Lambda unknown error!

After 2 days, there was my demo. I deployed my nodejs code on lambda function…

Curl - Your friend for Rest APIs/Calls - Basic Commands

Curl - Your friend for Rest APIs/Calls - Basic Commands

Curl is a wonderful tool for initiate REST APIs or calls. Or, you can literally…

Claude Code Skills — Build a Better Engineering Workflow with AI-Powered Code Reviews, Security Scans, and More

Claude Code Skills — Build a Better Engineering Workflow with AI-Powered Code Reviews, Security Scans, and More

Most developers use Claude Code like a search engine — ask a question, get an…

Building an AI Voicebot for Visitor Check-In — A Practical Guide to Handling the Messy Parts

Building an AI Voicebot for Visitor Check-In — A Practical Guide to Handling the Messy Parts

Every office lobby has the same problem: a visitor walks in, nobody’s at the…

Staff Engineer Study Plan for MAANG Interviews — The Complete 12-Week Roadmap

Staff Engineer Study Plan for MAANG Interviews — The Complete 12-Week Roadmap

If you’re a Senior Engineer (L5) preparing for Staff (L6+) roles at MAANG…

Latest Posts

Claude Code Skills — Build a Better Engineering Workflow with AI-Powered Code Reviews, Security Scans, and More

Claude Code Skills — Build a Better Engineering Workflow with AI-Powered Code Reviews, Security Scans, and More

Most developers use Claude Code like a search engine — ask a question, get an…

Building an AI Voicebot for Visitor Check-In — A Practical Guide to Handling the Messy Parts

Building an AI Voicebot for Visitor Check-In — A Practical Guide to Handling the Messy Parts

Every office lobby has the same problem: a visitor walks in, nobody’s at the…

Server Security Best Practices — Complete Hardening Guide for Production Systems

Server Security Best Practices — Complete Hardening Guide for Production Systems

Every breach post-mortem tells the same story: an unpatched service, a…

Staff Engineer Study Plan for MAANG Interviews — The Complete 12-Week Roadmap

Staff Engineer Study Plan for MAANG Interviews — The Complete 12-Week Roadmap

If you’re a Senior Engineer (L5) preparing for Staff (L6+) roles at MAANG…

XSS and CSRF Explained — The Complete Guide with Real Attack Examples and Defenses

XSS and CSRF Explained — The Complete Guide with Real Attack Examples and Defenses

XSS and CSRF have been in the OWASP Top 10 for over a decade. They’re among the…

OWASP Top 10 (2021) — Every Vulnerability Explained with Code

OWASP Top 10 (2021) — Every Vulnerability Explained with Code

The OWASP Top 10 is the industry standard for web application security risks. If…