Day 2 — Files & Directories

Day 1 Review

pwd — print working directory
ls, ls -lah — list files with options
cd, cd .., cd ~ — navigate directories
head, tail, less — view file contents
| pipe — chain commands; > redirect output to a file
* wildcard — match any number of characters

Setup — Fresh Data

We'll start with a fresh copy of the shell data today.

bash

$ cd ~/Desktop/
$ rm -r shell-lesson-data          # remove old copy
$ wget https://swcarpentry.github.io/shell-novice/data/shell-lesson-data.zip
$ unzip shell-lesson-data.zip
$ cd shell-lesson-data

Creating Directories and Files

bash

$ mkdir thesis         # make a new directory
$ ls -F

Practice: naming conventions

Try making a directory called "Thesis version1". What happens? Why? Remember: avoid spaces in file and directory names. Use underscores or hyphens instead.

bash

$ rm -r thesis version1   # -r flag required for directories (recursive)
$ mkdir thesis_v1
$ cd thesis_v1

Creating Files with nano

nano is a simple text editor that runs inside the terminal. Write something, then save with Ctrl+O and exit with Ctrl+X.

bash

$ nano draft.txt

File Extensions

bash

$ cp draft.txt draft.fasta
$ cp draft.txt draft.mp3
# Open these files — do they look any different?
# Extensions are just labels — Unix doesn't care what they are

Moving and Copying Files

bash

$ mkdir oldversions
$ mv draft.txt oldversions/      # move the file
$ cp oldversions/draft.txt .     # copy it back (. means "current directory")
$ nano draft.txt                 # edit the copy

Creating Files with echo

bash

$ echo This is my final draft
$ echo This is my final draft > thesis_final.txt    # write to file (overwrites)
$ echo This is my final draft >> thesis_final.txt   # append (adds to end)

Working with Multiple Files

bash

$ cd north-pacific-gyre
$ mkdir rawdata

# All of these do the same thing:
$ cp *.txt rawdata/
$ cp NEN* rawdata/
$ cp N*txt rawdata/

# Wildcards work on folders too
$ cp */*.txt all_text/

The ? Wildcard

? matches exactly one character — useful when you don't want to match everything.

bash

$ cd ../thesis
$ cp draft.txt draft_v1.txt
$ cp draft.txt draft_v2.txt
$ cp draft.txt draft_v3.txt
$ cp draft.txt draft_v11.txt
$ cp draft.txt draft_v12.txt

$ mkdir old
$ mv draft_v?.txt old   # matches v1, v2, v3 but NOT v11, v12 (two chars after v)
$ ls old

Counting, Sorting, and Combining

Navigate to exercise-data/alkanes — this directory has *.pdb files (atomic coordinate files from crystallography).

bash

$ wc -l *.pdb              # count lines in each file
$ wc *.pdb                 # lines, words, and characters
$ wc -l *.pdb > lengths.txt
$ less lengths.txt
$ sort -n lengths.txt      # sort numerically

Practice problem

Create a new version of lengths.txt that does NOT include the "total" line at the end. Hint: combine wc, head, and a pipe.

Answer: wc -l *.pdb | head -6 > lengths.txt

Extracting Columns with cut

bash

$ cd ../exercise-data/animal-counts
$ less animals.csv

$ cut -d , -f 2 animals.csv    # -d sets delimiter (comma), -f 2 = second column

# Remove duplicates — must sort first for uniq to work
$ cut -d , -f 2 animals.csv | sort | uniq
$ cut -d , -f 2 animals.csv | sort | uniq -c   # -c adds counts

File Permissions and Shell Scripts

bash

$ cd ../north-pacific-gyre
$ less goodiff.sh       # a shell script — do you recognize any commands?

# The first line (shebang) identifies the interpreter
$ which bash
$ which python
$ which R

$ ls -lh                # look at permissions (rw-r--r-- etc.)
$ chmod +x *.sh         # make scripts executable
$ ls -lh                # notice the x in permissions now
$ ./goodiff.sh          # run the script

Ask AI to Explain a Command AI

Combining commands with pipes produces powerful but hard-to-read one-liners. Use AI to understand them. Try this prompt:

Try this prompt in Claude or ChatGPT

I'm learning Unix shell commands. Can you explain what this command does, step by step? cut -d , -f 2 animals.csv | sort | uniq -c What does each part do? What would the output look like for a CSV file with animal names in the second column?

After you understand it, ask AI: "How would I modify this to show only animals that appear more than 3 times?"