How to Find and Copy Files in Linux

Post submitted by: at 8:18am

The amount of time spent trying to find, and then copy or move files around a windows machine or by doing it in a GUI takes a long time (when there are multiple files). In my case, I had to search through 10GB of data for anything with certain numbers associated with them. Grabbing all file extensions associated with the numbers and keeping the folder structure with it. We will use these numbers as an example:

These numbers are all in the file needToFind.txt
12345
67890
32145
12365

Lets assume a folder structure that has /mnt/shared/(year)/(month)/(day)/ . With the words in brackets being the actual numbers. The find command to just locate these files would be like this:

#read each number
while read line; do
        #look in everything under /mnt/shared/ for any file that starts with the number in needToFind.txt 
        find /mnt/shared/ -name $line*
done < needToFind.txt

Next we will copy these files to a mounted backup folder and keep the folder structure that they are located:

while read line; do 
        find /mnt/shared -name $line* -exec cp --parents {} /mnt/backup/ \;
done < needToFind.txt

To be able to monitor how your script is doing, you will want to add a few things. Since I was going through about 40,000 numbers in my “needToFind.txt”, I wanted some feedback on the terminal while it was running. To do this, I had it print our every 100th find request, and also print out the number of seconds it took to find and copy each file. This was just to run and to be periodically checked. Printing the amount of seconds each time was not necessary, but I wanted to see the average number of seconds it took to complete each number. The following shows how this was accomplished:

x=1
y=1
while read line; do
        t=$(date +"%s")
        if [[ $x == 100 ]]; then
                echo $y
                x=1
        fi
        find /mnt/shared/ -name $line* -exec cp --parents {} /mnt/backup/ \;
        x=$(($x + 1))
        y=$(($y + 1))
        now=$(date +"%s")
        sec=$(($now - $t))
        echo $sec
done < needToFind.txt

You can use find in many ways, and it can be very helpful. Please refer to the man pages for find (in the terminal, type man find) and have a look though it.


Theme: by Matthew Buchanan.