And, Ampersand, and & in Linux

[xSicKxBot]

And, Ampersand, and & in Linux

Take a look at the three previous articles, and you will see that understanding the glue that joins them together is as important as recognizing the tools themselves. Indeed, tools tend to be simple and understanding what mkdir, touch, and find do (make a new directory, update a file, and find a file in the directory tree, respectively) in isolation from each other is easy.

But understanding what

Code:

mkdir test_dir 2>/dev/null || touch images.txt && find . -iname "*jpg" > backup/dir/images.txt &

does, and why we would write a command line like that is a whole different story.

It pays to look more closely at the sign and symbols that live between the commands. It will not only help you better understand how things work, but will also make you more proficient in chaining commands together to create compound instructions that will help you work more efficiently.

In this article and the next, we’ll be looking at the the ampersand (

Code:

&

) and its close friend, the pipe (

Code:

|

), and see how they can mean different things in different contexts.

Behind the Scenes

Let’s start simple and see how you can use

Code:

&

as a way of pushing a command to the background. The instruction:

Code:

cp -R original/dir/ backup/dir/

Copies all the files and subdirectories in original/dir/ into backup/dir/. So far so simple. But if that turns out to be a lot of data, it could tie up your terminal for hours.

However, using:

Code:

cp -R original/dir/ backup/dir/ &

pushes the process to the background courtesy of the final

Code:

&

. This frees you to continue working on the same terminal or even to close the terminal and still let the process finish up. Do note, however, that if the process is asked to print stuff out to the standard output (like in the case of

Code:

echo

or

Code:

ls

), it will continue to do so, even though it is being executed in the background.

When you push a process into the background, Bash will print out a number. This number is the PID or the Process’ ID. Every process running on your Linux system has a unique process ID and you can use this ID to pause, resume, and terminate the process it refers to. This will become useful later.

In the meantime, there are a few tools you can use to manage your processes as long as you remain in the terminal from which you launched them:

• Code:

  jobs
  shows you the processes running in your current terminal, whether be it in the background or foreground. It also shows you a number associated with each job (different from the PID) that you can use to refer to each process:
  
  
  Code:

  $ jobs [1]- Running cp -i -R original/dir/* backup/dir/ & [2]+ Running find . -iname "*jpg" > backup/dir/images.txt &
  
  
• Code:

  fg
  brings a job from the background to the foreground so you can interact with it. You tell
  Code:

  fg
  which process you want to bring to the foreground with a percentage symbol (
  Code:

  %
  ) followed by the number associated with the job that
  Code:

  jobs
  gave you:
  
  
  Code:

  $ fg %1 # brings the cp job to the foreground cp -i -R original/dir/* backup/dir/
  
  If the job was stopped (see below),
  Code:

  fg
  will start it again.
  
  
• You can stop a job in the foreground by holding down [Ctrl] and pressing [Z]. This doesn’t abort the action, it pauses it. When you start it again with (
  Code:

  fg
  or
  Code:

  bg
  ) it will continue from where it left off…
  
  …Except for
  Code:

  sleep
  : the time a
  Code:

  sleep
  job is paused still counts once
  Code:

  sleep
  is resumed. This is because
  Code:

  sleep
  takes note of the clock time when it was started, not how long it was running. This means that if you run
  Code:

  sleep 30
  and pause it for more than 30 seconds, once you resume,
  Code:

  sleep
  will exit immediately.
  
  
• The
  Code:

  bg
  command pushes a job to the background and resumes it again if it was paused:
  
  Code:

  $ bg %1 [1]+ cp -i -R original/dir/* backup/dir/ &

As mentioned above, you won’t be able to use any of these commands if you close the terminal from which you launched the process or if you change to another terminal, even though the process will still continue working.

To manage background processes from another terminal you need another set of tools. For example, you can tell a process to stop from a a different terminal with the

Code:

kill

command:

Code:

kill -s STOP <PID>

And you know the PID because that is the number Bash gave you when you started the process with

Code:

&

, remember? Oh! You didn’t write it down? No problem. You can get the PID of any running process with the

Code:

ps

(short for processes) command. So, using

Code:

ps | grep cp

will show you all the processes containing the string “cp“, including the copying job we are using for our example. It will also show you the PID:

Code:

$ ps | grep cp 14444 pts/3 00:00:13 cp

In this case, the PID is 14444. and it means you can stop the background copying with:

Code:

kill -s STOP 14444

Note that

Code:

STOP

here does the same thing as [Ctrl] + [Z] above, that is, it pauses the execution of the process.

To start the paused process again, you can use the

Code:

CONT

signal:

Code:

kill -s CONT 14444

There is a good list of many of the main signals you can send a process here. According to that, if you wanted to terminate the process, not just pause it, you could do this:

Code:

kill -s TERM 14444

If the process refuses to exit, you can force it with:

Code:

kill -s KILL 14444

This is a bit dangerous, but very useful if a process has gone crazy and is eating up all your resources.

In any case, if you are not sure you have the correct PID, add the

Code:

x

option to

Code:

ps

:

Code:

$ ps x| grep cp 14444 pts/3 D 0:14 cp -i -R original/dir/Hols_2014.mp4   original/dir/Hols_2015.mp4 original/dir/Hols_2016.mp4   original/dir/Hols_2017.mp4 original/dir/Hols_2018.mp4 backup/dir/

And you should be able to see what process you need.

Finally, there is nifty tool that combines

Code:

ps

and

Code:

grep

all into one:

Code:

$ pgrep cp 8 18 19 26 33 40 47 54 61 72 88 96 136 339 6680 13735 14444

Lists all the PIDs of processes that contain the string “cp“.

In this case, it isn’t very helpful, but this…

Code:

$ pgrep -lx cp 14444 cp

… is much better.

In this case,

Code:

-l

tells

Code:

pgrep

to show you the name of the process and

Code:

-x

tells

Code:

pgrep

you want an exact match for the name of the command. If you want even more details, try

Code:

pgrep -ax command

.

Next time

Putting an

Code:

&

at the end of commands has helped us explain the rather useful concept of processes working in the background and foreground and how to manage them.

One last thing before we leave: processes running in the background are what are known as daemons in UNIX/Linux parlance. So, if you had heard the term before and wondered what they were, there you go.

As usual, there are more ways to use the ampersand within a command line, many of which have nothing to do with pushing processes into the background. To see what those uses are, we’ll be back next week with more on the matter.