How to loop through a file and perform an action per line
Written by Anthony Smith, 22 July 2020
Article was migrated from WordPress to Vanilla in March 2022
Sometimes you want to perform the same action multiple times with a single variable, that may be a vps ID, a process ID or some other unique identifier.
Here are a few examples of how you do that with a simple script, obviously there are other ways and probably better ways to do some of these things they are simply examples to work with.
Copy files of a certain type to a server
#!/bin/sh
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin
#find img files
find /images -type f -name "*.img" -exec basename '{}' \; > /tmp/imagelist
while read thing
do
scp /images/$thing [email protected]:/backups
done </tmp/imagelist
The above example finds any files with the .img extension in the /images directory and lists them in the file /tmp/imagelist the script then sets a variable of thing and performs an action for each thing which in this example it uses scp to move the .img files to the /backups directory on a remote server (the assumption is that the pub key for the source is installed on the remote server).
The line done </tmp/imagelistis what the script goes through 1 item at a time.
kill tor processes on a server
#!/bin/sh
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin
#find tor processes/pids
top -bc -n1 | grep torrc | awk '{print $1}' > /tmp/torpids
while read onion
do
kill -9 $onion
done </tmp/torpids
In the above example, the top command is used in batch mode and shows the full command line -bc and it runs only once -n1 then grp is used to find lines that contain torrc the awk prints the process id's only and finally they are sent to > /tmp/torpids
onion is used as the variable and then with extreme prejudice kill -9 $onion is used to nuke the tor processes, as before done </tmp/torpids is the list of process id's
shutdown all KVM guest servers
#!/bin/sh
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin
#get a list of all running kvm guests
virsh list | grep kvm | awk '{print $2}' >/tmp/guests
while read vm
do
virsh shutdown $vm
done < /tmp/guests
In the above example we use virsh list to get a list of all running guests | we grep for kvm to only get kvm guest id's/names (assuming your kvm guest servers use a naming convention with kvm at the start e.g. kvm12345) then use awk to print just the guest name and not the status and pass it through to /tmp/guests as a list
vm is the variable and while looping through /tmp/guests the action virsh shutdown is performed for each line.
The /tmp/guests file will look something like:
kvm123
kvm124
kvm125
kvm126
If you already have a list that you want to perform an action on you should be able to adapt any of the above to work with what you want to achieve.
Line by line for simplicity:
Line: while read set-a-variable-name-here
Explanation: This is where you decide what common variable you want to assign to the items in your list.
Line: do
Explanation: This is simply saying 'do the following'
Line: commands go here $some-variable
Explanation: You give the specific command and be sure to put the variable in the right place and use a $ to indicate that it is a variable, this must match the variable name you set on the first line.
Line: done < /tmp/list-of-stuff
Explanation: This reads the list of items into the script.
Any questions or comments are welcome.