Renaming files with non-interactive sftp

SFTP hangs around the IT Operations world like a bit of a bad smell.

Its pretty secure, it works, and its similar enough to FTP for software developers and business managers to understand, so its not uncommon to find it underpinning vast data transfer processes that have been designed in a hurry.

Of course, its very rudimentary in terms of what it can do, and very dependent on the underlying security of the OS on which it resides, so its not really something that should find a home in Enterprise IT solutions.

Anyway, sometimes you just have to deal with it. One problem that you will often encounter is that while you have SFTP access to a system, you may not have shell access via OpenSSH. This makes bulk operations on files a bit more difficult, but not impossible.

SFTP has a batch mode that allows you pass STDIN commands to the processor. If used in conjunction with non-interactive login (ie an OpenSSH Public/Private Key Pair) you can actually process bulk operations.

Let’s say you want to rename 500 files in a particular directory:

You can list the files as follows:

echo "ls -l1" | sftp -q -i ~/.ssh/id_rsa -b - user@sftp.mycompany.com:/dir1/

In this case, the parameter:

-b -

tells the processor to process the command coming from STDIN

You can now incorporate this into a BASH loop to complete the operation:

for f in `sftp -q -i ~/.ssh/id_rsa -b - user@sftp.mycompany.com:/dir1/ | grep -v sftp | grep -v Changing`;
    do
    echo "Renaming $f...";
    echo "rename $f $f.renamed" | sftp -q -i ~/.ssh/id_rsa -b - user@sftp.mycompany.com:/dir1/;
done