Intro to CLI and AV Tools


DPOE-N | 2021-06-24


with Brendan Coates (he|his)

About Me


gardener, edible microbe encourager, LATU member

Sr. Archivist, Academy Oral History Projects

Agenda


Part 1

History, Context, Objectives


12pm-12:50pm EDT/ 9am-9:50am PDT

Overview of the command line
Basic command line use
10min break

Part 2

Intro to Commands


1pm-2pm EDT / 10am-11am PDT

System info, navigation
inputs and outputs

LUNCH


2pm-3pm EDT / 11am-12pm PDT

Part 3

CLI Tools for DigiPres


3pm-3:50pm EDT / 12pm-12:50pm PDT

file movement
rsync, checksums, Bagit
10 minute break

Part 4

CLI Tools for AV


4pm-4:50pm EDT / 1pm-1:50pm PDT

structure of media files
Mediainfo, ffmpeg
10 minute break

Part 5

Intro to Scripting


5pm-6pm EDT / 2pm-3pm PDT

principles of automation
variables, loops, saving scripts

Overview


what are we doing here?
what is "open source"?
weren't those readings cool?

Introductions


What's your name/ pronouns/ location?
What do you do for work?
What do you do for fun?
What's 1 thing that brought you here?

What is the command line?

History

"terminal" connects to computer
UNIX -> Bourne shell -> GNU -> Bourne Again Shell (bash)
This technology stack is >30yrs old

Modern use

"terminal emulator"
runs scripts
server environments
ppl think you're a hacker

What good is it?

consistency | "meatdata"
doesn't get bored
historical connection
instantiate your workflows in code!

Basic Command Structure

command (options) [arguments]

Stopping commands

Ctl+c = stop
Ctl+z = kill

man pages

Navigation


PWD

"print working directory"

no flags for this

LS

"list" contents

flags:
-a = all
-l = long format
-1 = one per line

cd

"change directory"

special arguments:
~ = home folder
.. = parent directory

paths

absolute paths
from drive to file

relative paths
relative to where you are, now

pushd | popd

like cd but:
saves current dir

whereis

search environment variables

find where commands are installed

which

display command location

find

like ls but better
flags:
-type
-name
-iname

find

more flags:
-and
-or
* = wildcard

Users & Permissions


id

display user info

flags:
-n = name
-G = group

sudo

"super user do"
you're admin now

chmod|chown

chmod = change perms
chown = change ownership

ls -la

list directory perms

Inputs & Outputs


echo

print to terminal
text -> stdout

stdin

stdout

stderr

streams!

pipes

send output to command
common uses:
tail
less
grep (later)

redirects

send output to file
cmd >> log.txt
cmd >> /dev/null

&&

sequence commands
only runs cmd2 IF
cmd1 successful

touch

write blank file

cat

con"cat"enate files
reads files to stdout

all together

breakout rooms
1. write to a text file
2. write dir contents

lunch

see you in an hr!

CLI for DigiPres

moving files
doing hashes
BagIt

Moving Files


cp

"copy"
flags:
-r = recursive (subdirs)
-a = timestamps
-i/n = interactive/not
-v = verbose

mv

"move"
renames files/paths
flags:
-i/n = interactive/not

mkdir

"make directory"
flags:
-p = parents
-v = verbose

rsync

sync files/ dirs
flags:
-a = archive mode
-v = verbose
-z = compress
-x = extended attrs

rsync

flags:
--progress
--logfile=path
--exclude

Checksums


Hashes

break file into blocks
read blocks + compute hash
check full file hash

CRC

break file into blocks
read blocks + compute check value
check block hashes +
block position

md5sum

generate MD5 hash
flags:
-c = check files

shasum

generate SHA hash
flags:
-a = algorithm
-p = portable
-c = check files

BagIt

Library of Congress + IETF standard
for folder architecture

supports robust verification
of package contents + integrity

bagit-python

ask Nick and Ben abt this tmrw

grabbags

wrapper for bagit-python
maintianed by AMIA OSC
exposes bagit-python to CLI

Tools for AVPres

media file structure

wrapper / container
the file header and structure
stream
individual media bitstream
streams are wrapped into files
nougat | peanut |caramel
video | audio |subtitles
codec
stands for "encode/decode"
the method/ algorithm
for en/decoding data
Snickers
encoding = making a Snickers
decoding = eating it
codec = laws of chemistry

Tools for AVPres

MediaInfo

ffmpeg

Scripting

putting everything together

CLI scripting

find -exec

execute a command
on output of find

FOR loops

iterate through something
loop thru directory

variables

variable=value
${variable} - deploy it

variable substitution

file="/path/to/file.mkv"
${file%/.} = ?
${file%.mkv}.mp4 = ?
ok let's actually write a lil script

questions?

thank you!

and thanks to everyone who made this possible