How to transfer files from EC2 to your local without ssh?

Arghya Saha -

One of the most common situations while working with remote servers is to transfer huge files from the remote server to our local machine for better debugging.

We all have gone through the issue where you cannot directly ssh into the remote machine, i.e. you have some kind of jump box server or bastion which might make your life more difficult. Then we have to upload the files to some S3 server and then again download them, but maybe to make matters worse, the remote server might not have access to upload files.

So what can you do now? This is beta/stage/pre-production environment so you want to quickly get the file and debug the issue, rather than spend hours on getting access to the file. You can use tunneling to solve your problem, here is how-

DONOT DO THIS ON PRODUCTION, TO AVOID SECURITY RISK.
  • First, you need to have python or any other language web server, I prefer python since all Linux distributions have python installed and the syntax is also easy.
  • Go to the folder where you have the file which you need in your local machine
  • Run the below command. This would create an HTTP server in port 3333 with all the files listed. You can check this in your local system on how it looks.
python3 -m http.server 3333
  • Now your python file host server is running in the remote server but is still inaccessible from outside. To make your remote server accessible to the outside world, we shall use a service that can expose the localhost port. localtunnel, ngrok are few of the free/paid services which can help you do so. We shall use localtunnel for this purpose.
GitHub - localtunnel/localtunnel: expose yourself
expose yourself. Contribute to localtunnel/localtunnel development by creating an account on GitHub.
GitHublocaltunnel
  • Install localtunnel in your remote server
npm install -g localtunnel
  • Run the following command, where 3333 is the port of the python server. 
lt -p 3333
  • You would see an output like this
  • Once you open the link on the browser you shall see all the content of the folder where you are hosting the python server. Now you can simply browse all the files and folders
Do remember to stop both python and localtunnel server.

PS - This is applicable for any kind of server be it AWS, GCP, AZURE, Digital Ocean, or any other cloud computing server, which has outbound access to the internet, if you don't have outbound access, then you are on the hard luck side and the only way would be a multi-step process that others follow in your team.