System admin¶
Packages¶
Web servers and monitoring¶
caddy
certbot
firefox
monit
kibana
netdata
elasticsearch
fluentd
sudo netdata -c ~/elk/netdata.conf
fzf
htop
jq
ag
jq
rxvt-unicode
sqlite
tdns-cli
gotools
openssl
autoconf
automake
pkg-config
readline
glibc
ncurses
clang
go
openjdk
vim
expect
sudo usermod -aG docker $USER
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
sudo apt-get update
sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get autoremove
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu (lsb_release -cs) stable"
sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common
sudo apt dist-upgrade
sudo apt-get install unattended-upgrades
sudo apt-key fingerprint 0EBFCD88
sudo apt-get install sqlite3
sudo apt-get install build-essential
sudo apt-get install python3
sudo apt-get install npm
sudo apt-get install caddy
sudo apt-get install chkrootkit
sudo apt-get install console-data
sudo apt-get install docker-ce docker-ce-cli containerd.io
sudo apt-get install fail2ban
sudo apt-get install git
sudo apt-get install golang
sudo apt-get install libpam
sudo apt-get install mosh
sudo apt-get install mosh-server
sudo apt-get install nodejs
sudo apt-get install pam
sudo apt-get install pip
sudo apt-get install pip3
sudo apt-get install python-pip3
sudo apt-get install python3-pip
sudo apt-get install rspamd
sudo apt-get install socat
sudo apt-get install sqlite3
Shell Stuff¶
basic shell commands
chs || sh
#-v for verbose
#-x shows each slice of the process too preceeded by a plus
#-n for parsing out errors
#-e to exit after one error
source
xargs
export setenv
hash, rehash
history
!!
!!:n,!!:^,!!:$,!!:*,
!vi
!?vi
!!:s/
pushd
popd
dirs
alias
unalias
set
unset
limit
wc
echo
read
print
cat
nc
tr
shift
head
tail
sort
uniq
fmt
script
fold
col
roff
expand
colcrt
more
expr
eval
test, [
dc
bc
fstat
umask
newgrp
mktemp
hostname
uname
realpath
basename
dirname
banner
wall
motd
date
calender
times
sed -- stream pattern matcher
awk -- database pattern matcher
grep-- regexp pattern matcher
find-- reexp filematcher
mtree
cut
paste
copy
diff
patch
od
{ }, ( )
if [ ] #-f , -d
then
elif
then
else
fi
for i in
do
done
while
do
done
case
)
)
)
esac
\
break
continue
routine()
{
}
# a function can also returrn a value. else it returns the last most
return value
# all variables are global. local variables must be preofixed by the
word local set
exit
trap
. filename for executing another command
$i ->arguments
$# ->parameter count
$@ ->all parameters
$$ ->PID
$- ->shell invocation flags
${name1*name2} conditional substiution
-lt -gt -o -a -ne = set
> redirect stdout to file
>& redirect stdout and file
>>append to file
<, <&
<< read till word after $ expansion is done on the word
| sort of like ><
user management
pw
adduser
chsh
kernel management
halt reboot shutdown nextboot fastbooot fasthalt dmesg
kenv sysctl savecore kldload kldunload kldconfig kldstat loader
kdump ktrace kdbg
harddisk management
mount_* umount
sync
g*
swapon swapoff
dumpfs
fdisk newfs disklabel
fsck badsect
tunefs
atacontrol
cdcontrol
swapinfo
mdconfig -- virtual disk management
ldconfig -- library management
adjkerntz
conscontrol
vidcontrol
kbdcontrol
camcontrol
mixer
devd
devfs
scanpci
devinfo
networking
ifconfig
sockstat
routed
ipf pfctl pflogd
dhclient, script
ping
traceroute
whois
dig
art
mesg
write
talk
login
nologin
getty
stty
mail
script
nfsstat
ipfw
daemons
nfsd
sshd
ftpd
telentd
talkd
crond
filemanagement
cd
pwd
cpio
rm
rmdir
mkdir
ls
mv
ln
unlink
du
df
md5
sha256
sha1
md5
ar
tar
bz2
compress
pax
rdump rrestore
help
man
whatis
apropos
ed
processs management
at
ps
vmstat
top
sleep
kill
pkill
who
fg
fc
exec
mkfifo
nice
watch
wait
xserver xhost, xauth, xdm done
kerberos
rlogind
telnetd
jaild
nfsd
lpd
crond
httpd, httpc
httptunneld, httptunnelclient
sendmail
tracker
squid
squidguard
dhcpd
named
routed
ipfilter
inetd
irc
jabber
dc
cvsd svnd
xhost + machinn will give machine the access to your display
/etc/Xn.hosts file is to store the common machines
xauth will start a xauth session
add machine protocol hexkey
extract file machine
generate machine protocol [trusted|untrusted|timeout] [group groupid -- that the machines will be displayed here]
list
merge filename
remove
exit
info info about the xauth file
quit
help
?
machine:0 implies the key this xserver gives to machine
machine/unix:0 implies the key this xserver requires
xfwp
proymngr
xfindproxy are used to add another layer for this security
xmodmap
adduser
rmuser
chmod
chown
chpass
pw
stickybit->deleting only by owner or root in th dir
retain the executable image in memory even after exit
setuidbit->for su and executing as owner
setgidbit-> " group member
for directories implies everything within this directory would be the
owners || the groups
who to display current users onn thee system
w
users
pw also has lock unlock abilities
to kick user, kill his lgin process
LXC¶
https://askubuntu.com/questions/1048217/how-do-i-copy-a-file-directory-from-host-into-a-lxd-container#1051763
https://gist.github.com/ammgws/381b4d9104c4e2b43b9210f33f03a15a
https://lxdware.com/forwarding-host-ports-to-lxd-instances/
https://medium.com/@tcij1013/lxc-lxd-cheetsheet-effb5389922d
https://unamelinux.com/posts/lxc-and-lxd/
https://www.unix.com/solaris/186001-how-do-i-mount-zfs-snapshot.html
sudo apt install lxd lxd-client
sudo lxd init
sudo lxc storage list
sudo lxc image list
sudo lxc image list images:
sudo lxc image list images:ubuntu
sudo lxc launch ubuntu:20.04
sudo lxc list
sudo lxc exec rapid-foal bash
lxc remote list
lxc storage list
lxc profile create plannr-port
lxc profile add rapid-foal plannr-port
lxc profile device add rapid-foal plannr-port proxy connect="tcp:127.0.0.1:8000" listen="tcp:0.0.0.0:8000"
sudo lxc profile show plannr-port
Useful commands¶
sudo reboot
nohup fossil server --https . &
rsync -rv --exclude=.git birthyrhm sites/bitrhythm
locale-gen UTF-8
locale-gen en_US.UTF-8
dpkg-reconfigure locales
echo $LANG
echo $LC_CTYPE
lsof -Pnl +M -i4 | grep 8015
cat /etc/os-release
df -h
journalctl _SYSTEMD_UNIT=sshd.service | egrep "Failed|Failure"
sudo journalctl --vacuum-time=1w
sudo apt-get autoclean
sudo apt-get autoremove
sudo apt-get autoremove --purge
sudo systemctl status fail2ban
sudo sqlite3 /var/lib/fail2ban/fail2ban.sqlite3 "select distinct ip from bans"
~/go/bin/croc # for p2p file transfers
tmux new -s main
tail -f caddy.log
Tuning¶
sysctl -w net.ipv4.tcp_sack=0
sysctl kern.ipc.maxsockbuf=16777216
sysctl kern.ipc.somaxconn=1024
sysctl net.inet.tcp.recvbuf_inc=262144
sysctl net.inet.tcp.recvbuf_max=16777216
sysctl net.inet.tcp.recvspace=262144
sysctl net.inet.tcp.sendbuf_inc=262144
sysctl net.inet.tcp.sendbuf_max=16777216
sysctl net.inet.tcp.sendspace=262144
sysctl net.inet.tcp.syncookies=0
Firewall¶
https://www.freecodecamp.org/news/securing-your-linux-web-server/
sysctl -w net.ipv4.ip_unprivileged_port_start=0
sudo ufw allow 30025
sudo ufw allow 25
sudo ufw allow 443
sudo ufw allow 4567
sudo ufw allow 465
sudo ufw allow 465 587
sudo ufw allow 587
sudo ufw allow 6080
sudo ufw allow 80
sudo ufw allow 993
sudo ufw default allow outgoing
sudo ufw default deny incoming
sudo ufw delete 11
sudo ufw delete 11 14
sudo ufw delete 13
sudo ufw delete 14
sudo ufw delete 4
sudo ufw delete 5
sudo ufw delete 6
sudo ufw delete <no>
sudo ufw enable
sudo ufw status
sudo ufw status numbered
Maddy¶
sudo opendkim-genkey -b 2048 -d domain -D /etc/letsencrypt/dkim/domain -v
cat /var/lib/maddy/dkim_keys/xyzzyapps.link_default.key
curl https://get.acme.sh | sh -s email=email@email.com
mkdir -p /etc/maddy/certs/mx1.xyzzyapps.link
docker pull wordpress
docker run -d -p 8000:8000 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer
docker volume create portainer_data
docker-compose up -d
maddyctl
maddyctl creds
maddyctl creds list
maddyctl creds create test@xyzzyapps.link
maddyctl creds remove test@xyzzyapps.link
maddyctl imap-acct
maddyctl imap-acct --help
maddyctl imap-acct create test@xyzzyapps.link
maddyctl imap-acct list
maddyctl imap-acct remove test
maddyctl imap-acct remove test@xyzzyapps.link
maddyctl status
sudo groupadd maddy
sudo useradd -m maddy
sudo journalctl -u maddy
sudo ls -R /etc/systemd | grep maddy
sudo lsof -i -P -n | grep LISTEN
sudo maddy
sudo maddyctl creds create test@xyzzyapps.link
sudo mkdir -p /etc/maddy/certs/mx1.xyzzyapps.link
sudo mv /usr/local/bin/maddy /usr/bin/maddy
sudo mv /usr/local/bin/maddyctl /usr/bin/
sudo rm -rf /var/lib/maddy/
sudo rm /var/lib/maddy/credentials.db
sudo rm /var/lib/maddy/imapsql.db*
sudo systemd-analyze verify maddy
systemctl start maddy
sudo ~/.acme.sh/acme.sh --install-cert -d "xyzzyapps.link" --fullchain-file /etc/maddy/certs/mx1.xyzzyapps.link/fullchain.pem
~/.acme.sh/acme.sh --issue -d xyzzyapps.link -d "*.xyzzyapps.link" --dns --yes-I-know-dns-manual-mode-enough-go-ahead-please --renew
SSH¶
sudo service mosh-server
SSH - disable root login
xrdp - remote display
Caddy¶
I moved onto Caddy from nginx for serving simple sites
sudo sysctl -w net.ipv4.ip_unprivileged_port_start=0
sudo setcap 'cap_net_bind_service=+ep' /home/bin/caddy
caddy hash-password
Caddy Condig¶
(logging) {
encode gzip
log {
output file /home/user/caddy.log {
roll_disabled
}
format json
}
}
www.site.com {
import logging
root * ./www.site.com
file_server
}
site.com {
import logging
root * ./www.site.com
file_server
}
service.site.com {
import logging
reverse_proxy 127.0.0.1:3000
redir /path /path/
basicauth /path {
Bob JDJhJDEwJFZTSWxlaFhHTWljbTlXZHJtVzJGN2VwU0VtU3FuU3VVSWtsZ2c3OTVNOER5dmZISEsyRWMu
}
handle /themes/* {
root * /home/user/folder
file_server
}
handle /* {
root * /home/user/folder
php_fastcgi 127.0.0.1:9000
}
}
Supervisor config¶
./supervisord -c ~/supervisor.conf
[program-default]
autorestart=true
[program:stuff]
command=/home/project/env/bin/python server.hy
directory=/home/project
[program:caddy]
command=/home/caddy/bin/caddy run
directory=/home/sites/
[supervisord]
logfile = ./supervisor.log
Robots
User-agent: ia_archiver
Disallow: /
Git¶
git is the opposite of svn
-it is fundamentally different
-the terms used by git and svn are common which makes it difficult
workflows are different
more workflow options are possible in git and using git as an svn repository is going to be problematic
permission vs consensus
top-down vs bottum-up
Git is not a version control system. It is a file system.
Git provides versioning just like ZFS provides snapshots.
Git stores two files but enough information to recreate the previous file at demand.
For a given directory of source code, git does the same to each and every file including the directory itself.
recommended workflow
0) branch = local branch + remote branch
every author should have a branch
every branch formed or deleted so as to have a unique set of requirements
merging happens by bottom up merges by the merge master if possible
mkdir foo
git init
git clone
git add
git rm
git fetch
git stash
git prune
git diff
git checkout
git cherry-pick
git merge
git rebase
git status
git commit
git branch
git patch-set
branch spec / ref spec
Hosting¶
EC2 / DO OpenVz
Cloud based¶
Python anywhere
Heroku
https://sandstorm.io/
https://cloudron.io/
https://caprover.com/
https://www.evennode.com/pricing
https://www.nodechef.com
Cpanel¶
https://freedombox.org/
Yunohost
Webmin
Wordpress¶
# BEGIN WordPress
# The directives (lines) between "BEGIN WordPress" and "END WordPress" are
# dynamically generated, and should only be modified via WordPress filters.
# Any changes to the directives between these markers will be overwritten.
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress