Advanced Bash-Scripting HOWTO

A guide to shell scripting, using Bash

Mendel Cooper

thegrendel@theriver.com

This is a major update on version 0.2. -- more bugs swatted, plus much additional material and example scripts added. This project has now reached the proportions of an entire book. See NEWS for a revision history.

This document is both a tutorial and a reference on shell scripting with Bash. It assumes no previous knowledge of scripting or programming, but progresses rapidly toward an intermediate/advanced level of instruction. The exercises and heavily-commented examples invite active reader participation. Still, it is a work in progress. The intention is to add much supplementary material in future updates to this HOWTO, so that it will gradually evolve into an LDP "guide", i.e., a complete book.

The latest version of this document, as an archived "tarball" including both the SGML source and rendered HTML, may be downloaded here from the author's home site.

Table of Contents

1. Why Shell Programming?

2. Starting Off With a Sha-Bang

2.1. Invoking the script
2.2. Shell wrapper, self-executing script

3. Tutorial / Reference

3.1. exit and exit status
3.2. Special characters used in shell scripts
3.3. Introduction to Variables and Parameters
3.4. Quoting
3.5. Tests
3.6. Operations and Related Topics
3.7. Variables Revisited
3.8. Loops
3.9. Internal Commands and Builtins
3.10. External Filters, Programs and Commands
3.11. System and Administrative Commands
3.12. Backticks (`COMMAND`)
3.13. I/O Redirection
3.14. Recess Time
3.15. Regular Expressions
3.16. Subshells
3.17. Restricted Shells
3.18. Process Substitution
3.19. Functions
3.20. Aliases
3.21. List Constructs
3.22. Arrays
3.23. Files
3.24. Here Documents
3.25. Of Zeros and Nulls
3.26. Debugging
3.27. Options
3.28. Gotchas
3.29. Miscellany
3.30. Bash, version 2

4. Credits

Bibliography

A. Contributed Scripts

B. A Sed and Awk Micro-Primer

B.1. Sed
B.2. Awk

C. A Sample .bashrc File

D. Copyright

List of Tables
3-1. bash options
B-1. sed operators
B-2. examples

List of Examples
2-1. cleanup: A script to clean up the log files in /var/log
2-2. cleanup: An enhanced and generalized version of above script.
2-3. shell wrapper
2-4. A slightly more complex shell wrapper
2-5. A shell wrapper around an awk script
2-6. Perl embedded in a bash script
3-1. exit / exit status
3-2. Code blocks and I/O redirection
3-3. Saving the results of a code block to a file
3-4. Backup of all files changed in last day
3-5. Variable assignment and substitution
3-6. Using param substitution and :
3-7. Renaming file extensions:
3-8. Using pattern matching to parse arbitrary strings
3-9. What is truth?
3-10. Equivalence of [ ] and test
3-11. Tests, command chaining, redirection
3-12. arithmetic and string comparisons
3-13. testing whether a string is null
3-14. zmost
3-15. Compound Condition Tests Using && and ||
3-16. Representation of numerical constants:
3-17. arglist: Listing arguments with $* and $@
3-18. Variable Assignment
3-19. Variable Assignment, plain and fancy
3-20. Positional Parameters
3-21. wh, whois domain name lookup
3-22. Using shift
3-23. Using declare to type variables
3-24. Indirect References
3-25. Generating random numbers
3-26. Simple for loops
3-27. for loop with two parameters in each [list] element
3-28. Missing in [list] in a for loop
3-29. Using efax in batch mode
3-30. Simple while loop
3-31. Another while loop
3-32. while loop with multiple conditions
3-33. until loop
3-34. Effects of break and continue in a loop
3-35. Using case
3-36. Creating menus using case
3-37. Creating menus using select
3-38. Creating menus using select in a function
3-39. Using getopts to read the flags/options passed to a script
3-40. Using set with positional parameters
3-41. basename and dirname
3-42. Variable assignment, using read
3-43. Changing the current working directory
3-44. "Including" a data file
3-45. Waiting for a process to finish before proceeding
3-46. Using ls to create a table of contents for burning a CDR disk
3-47. Badname, eliminate file names in current directory containing bad characters and white space.
3-48. Log file using xargs to monitor system log
3-49. copydir, copying files in current directory to another, using xargs
3-50. Showing the effect of eval
3-51. Forcing a log-off
3-52. Using expr
3-53. Letting let do some arithmetic.
3-54. Using date
3-55. Emulating "grep" in a script
3-56. Checking words in a list for validity
3-57. toupper: Transforms a file to all uppercase.
3-58. lowercase: Changes all filenames in working directory to lowercase.
3-59. rot13: rot13, ultra-weak encryption.
3-60. Using column to format a directory listing
3-61. nl: A self-numbering script.
3-62. Formatted file listing.
3-63. printf in action
3-64. Using cpio to move a directory tree
3-65. stripping comments from C program files
3-66. uudecoding encoded files
3-67. Using seq to generate loop arguments
3-68. secret password: Turning off terminal echoing
3-69. pidof helps kill a process
3-70. Effects of exec
3-71. killall, from /etc/rc.d/init.d
3-72. Redirecting stdin using exec
3-73. Redirected while loop
3-74. Redirected until loop
3-75. Redirected for loop
3-76. Redirected if/then test
3-77. Variable scope in a subshell
3-78. Running parallel processes in subshells
3-79. Running a script in restricted mode
3-80. Simple function
3-81. Function Taking Parameters
3-82. Converting numbers to Roman numerals
3-83. Local variable visibility
3-84. Recursion, using a local variable
3-85. Aliases within a script
3-86. unalias: Setting and unsetting an alias
3-87. Using an "and list" to test for command-line arguments
3-88. Using "or lists" in combination with an "and list"
3-89. Simple array usage
3-90. Some special properties of arrays
3-91. An old friend: The Bubble Sort
3-92. Complex array application: Sieve of Erastosthenes
3-93. dummyfile: Creates a 2-line dummy file
3-94. broadcast: Sends message to everyone logged in
3-95. Multi-line message using cat
3-96. upload: Uploads a file pair to "Sunsite" incoming directory
3-97. Setting up a swapfile using /dev/zero
3-98. test23, a buggy script
3-99. test24, another buggy script
3-100. Trapping at exit
3-101. Cleaning up after Control-C
3-102. String expansion
3-103. Indirect variable references - the new way
3-104. Using arrays and other miscellaneous trickery to deal four random hands from a deck of cards
A-1. manview: A script for viewing formatted man pages
A-2. rn: A simple-minded file rename utility
A-3. encryptedpw: A script for uploading to an ftp site, using a locally encrypted password
A-4. behead: A script for removing mail and news message headers
A-5. ftpget: A script for downloading files via ftp
A-6. password: A script for generating random 8-character passwords
A-7. fifo: A script for making daily backups, using named pipes
A-8. tree: A script for displaying a directory tree
C-1. Sample .bashrc file