As part of this bash script I worked on recently I have been playing a bit with cut and tr. These are old Unix commands and I haven’t used them that much. It’s mostly been a bit of sed or awk (am no expert in either of them) but as part of creating this script I was re-introduced to cut and tr when Googling for some solution.
The cut command cuts text along a delimiter and can output the fields you choose. The tr command can translate text.
This is not tutorial but here’s an example of where I used these two today. I have a bunch of docker volumes I want to list just the names of. The default output is thus:
|
1 2 3 4 5 6 |
$ docker volume ls DRIVER VOLUME NAME local kea-knot_keaconfig local kea-knot_kealeases local kea-knot_knotconfig local kea-knot_knotzones |
Sure, I can pipe a grep kea-knot to get just the lines with volume names, but how can I extract just the volume name? This is where cut comes into play. The following should in theory get me the second column:
|
1 |
docker volume ls | grep kea-knot | cut -d ' ' -f 2 |
This does not however. It only returns a bunch of blank lines. Why? Because I am telling it to cut along a space, and between local and the kea-knot_keaconfig text for instance there’s multiple spaces. This the second column is actually the fifth or sixth column.
So what I need here is a way to trim the spaces (or get cut to match on one or more spaces – it doesn’t seem to do that). Here’s where its buddy tr comes into play. tr can also squeeze characters. So if I do tr -s ' ' it will squeeze multiple spaces into a single one. And you can guess where the story goes from here… I pass that to cut who will dutifully get me the second field as expected. :)
|
1 2 3 4 5 |
$ docker volume ls | grep kea-knot | tr -s ' ' | cut -d ' ' -f 2 kea-knot_keaconfig kea-knot_kealeases kea-knot_knotconfig kea-knot_knotzones |
Tada!
In this specific case however if I had read the docker volume command docs a bit more I would have discovered that adding a --quiet switch to docker volume ls would have got me just the volume names!
|
1 2 3 4 5 |
$ docker volume ls --quiet | grep kea-knot kea-knot_keaconfig kea-knot_kealeases kea-knot_knotconfig kea-knot_knotzones |
Oh well. Wouldn’t have had a bit of fun with cut and tr then would I? :o)
Anyways, why was I doing this? Because I wanted to move multiple volumes from one Docker host to another. I had blogged about a single volume previously, now I loop over that:
|
1 2 3 |
for VOLUME in `docker volume ls --quiet | grep kea-knot`; do docker run --rm -v $VOLUME:/$VOLUME alpine tar -czv --to-stdout -C /$VOLUME . > $VOLUME.tgz ; done |
Copy these over to the destination host and loop over to import them:
|
1 2 3 |
for VOLUME in kea-knot*.tgz; do cat $VOLUME | docker run --rm -i -v ${VOLUME%.tgz}:/${VOLUME%.tgz} alpine tar xzf - -C /${VOLUME%.tgz} ; done |
Updates: I will add to this blog post as I find more use cases & examples.
Random Numbers
I want to generate a 20 digit random number (to use with an Azure Storage account). /dev/random will give random characters. So pipe it to tr and tell it to ignore anything that’s not a number.
Start with this:
|
1 |
LC_CTYPE=C tr -dc [:digit:] < /dev/urandom |
I am not a 100% switched on about LC_CTYPE. Without it tr gives an tr: Illegal byte sequence error. Setting it tells tr what locale to use to interpret the output of /dev/urandom:
LC_CTYPE This variable determines the locale category for character handling functions, such as tolower(), toupper() and isalpha(). This environment variable determines the interpretation of sequences of bytes of text data as characters (for example, single- as opposed to multi-byte characters), the classification of characters (for example, alpha, digit, graph) and the behaviour of character classes. Additional semantics of this variable, if any, are implementation-dependent.
So tr takes the output of /dev/random. The -d switch tells it to delete anything not matching the class I specify [:digit:] (which stands for all numbers). However the -c switch says complement the class I have specified – thus -c [:digit:] says anything which is not a number – and so -dc [:digit:] says delete anything which is not a number.
The end result is I end up with a bunch of random number!
|
1 |
1059421075805816656509968068170170479138516931918797656798804230515413726340104282755520093984753569078069447716927631933211349433244211621041307320305965249530686753190898357446297864797938577884946531572213728446695783008651403843648599273206... |
I don’t want so many random numbers, just 20. So I pipe that to head to get just that many:
|
1 |
LC_CTYPE=C tr -dc [:digit:] < /dev/urandom | head -c 20 |
If I wanted a mix of alphabets and numbers instead I can use the following:
|
1 |
LC_CTYPE=C tr -dc [:alnum:] < /dev/urandom | head -c 20 |
Or for just alphabets:
|
1 |
LC_CTYPE=C tr -dc [:alpha:] < /dev/urandom | head -c 20 |
To give credit where due the above is inspired from this gist. Do check that for more examples.