Millet Porridge

English version of https://corvo.myseu.cn

0%

Some Bash Tricks (Updated Irregularly)

Posted on Edited on In Disqus:
This blog records some Bash scripts I've encountered. The purpose is absolutely not to encourage anyone to use shell scripts (quite the opposite) — only to show that shell scripts can do many things.

This blog records implementation methods from Bash scripts I’ve encountered, as an excerpt and summary. The purpose of writing it is absolutely not to encourage everyone to use shell scripts (quite the opposite) — only to show that shell scripts can do many operations.

bash two-dimensional arrays

Bash itself does not support two-dimensional arrays; this method is just a substitute.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
test_version_arrays=(
    'pkg_ver=(python2 redis==2.10.5     rq==0.6)'
    'pkg_ver=(python2 redis==2.10.5     rq==0.9)'
    'pkg_ver=(python2 redis==2.10.5     rq==0.13)'
    'pkg_ver=(python2 redis==3.2.1      rq==0.13)'

    'pkg_ver=(python3 redis==3.2.1   rq==0.13)'
    'pkg_ver=(python3 redis==3.2.1   rq==1.0)'
)

for elt in "${test_version_arrays[@]}"
do
    # evaluate each array
    eval $elt;
    for pkg in "${pkg_ver[@]:1}"
    do
        echo $pkg
    done
done

Printing indices in a for loop

Copy from: https://stackoverflow.com/a/6723516/5563477

1
2
3
4
foo=('s1' 's2' 's3')
for i in "${!foo[@]}"; do
  printf "%s\t%s\n" "$i" "${foo[$i]}"
done

Deduplicating strings by md5 value

If we want to deduplicate and save many strings, the first method that comes to mind is deduplicating the strings first, then saving. Here we take a different approach: save the files first, then deduplicate the files.

1
2
3
4
5
6
7
8
9
10
11
12
13
arr=('str1' 'str1' 'str1' 'str1' 'str2' 'str3')

mkdir -pv tmp

for i in "${!arr[@]}"; do
    echo "${arr[$i]}" > tmp/$i.txt
done

# md5sum all files, sort, and find adjacent identical hashes
md5sum tmp/* | \
sort | \
awk 'BEGIN{lasthash = ""} $1 == lasthash {print $2} {lasthash = $1}' |\
xargs rm -v

Sorting part of an array’s values with sed and sort

1
2
3
4
5
6
arr=('version-1' 'version-2' 'version-3' 'version-39' 'version-13')

# first use sed to extract the num from version-<num>, converting to <num>@version-<num> form, then use sort on the value before @
data=($(for a in "${arr[@]}"; do echo "$a" | sed 's/.*version-\([0-9]*\)/\1@&/'; done | sort -t @ -k 1 -nr))

echo ${data[@]}