# Docker Talker
### Overview of Docker & Kubernetes + Ideas for Future Usage
by
[traceypooh](https://twitter.com/tracey_pooh)
June 2017
_ESC_ for slide overview
---
### Meta
- Docker - ~15min
- Kubernetes - ~15min
- Proposing more docker - ~15min
- Open Issues / Brainstorm / Discusion / Questions - ~15min
---
# Act I - Setting
---
### Docker
https://www.docker.com/docker-mac
- mac: hypervisor (pthread per vCPU)
- linux: lxc
- windows: hyper-v
- AWS
- GCE / Google Compute Platform
### Key Concepts:
- _images_ readonly
- _containers_ **copy-on-write** layers; writes over first relevant layer
---
### Docker Advantages:
- 4-6x times # of docker servers as KVM VMs
- Google search, gmail, Docs, etc -- containers
- docker **just abstracts the OS kernel** -- not the entire device (VMs)
- local dev environments exactly like live server
- test/run multiple dev environments & versions -- all on same host
- reliable, rebuildable entire recipe for re/building
---
### Yah, but why?
---
### need to build a binary?
docker it!
### on a plane w/o network?
docker it!
### debug a server?
docker it!
### tweak anything petabox / website dev?
docker it!
### derive some files?
slap a docker sticker on it!
---
[image missing xxx]
---
### Docker examples
run xenial:
``` bash
docker run --rm -it ubuntu:latest bash
```
run zesty:
``` bash
docker run --rm -it ubuntu:rolling bash
```
_this talk_:
``` bash
git clone https://github.com/traceypooh/docker-talker; open docker-talker/index.html
```
---
### Docker resources
- docker best practices
- https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/
- misc
- http://www.zdnet.com/article/what-is-docker-and-why-is-it-so-darn-popular/
---
### Kubernetes (greek: helmsman/pilot)
open-source system for automating deployment, scaling and management of containerized applications that was originally designed by Google and donated to the Cloud Native Computing Foundation.
It aims to provide a "platform for automating deployment, scaling, and operations of application containers across clusters of hosts"
---
### Kubernetes gist
- pods of containers
- can share resources among whole group
- each pod runs on on physical machine
- so all up / all down on machine, by design
- can share folders and folders w/ tcp/ip connections between them
- BFFs:
- kubeadm (cluster)
- minikube (mac)
---
### Kubernetes
- scheduling
- load / resource balancing / high availability
- versioning and rolling rollout
- ... with rollback
- secrets
- can "prefer" placement, when avail
- eg: derive on primary if resources avail
- saves network copying
- else derive like normal, remote, copy item down first..
---
### pods, services, controllers -- o mai!
#### _Excellent Overview_ - concepts, history, rationale:
- https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ \*
- https://coreos.com/kubernetes/docs/latest/services.html
\* and Jeopardy! answer ;-)
---
# Demo Time!
---
### K8 Demo Time
- using _kubeadm_ tool - 1 master; 3 nodes
- kallr
- ```watch -dc -n 1 kubectl get deploy,rs,pods,services --all-namespaces```
- kube-dev1
- kubeadm-setup-master
- kwebnode
- kwebnode-scale
- kube-dev2..4
- kubeadm-setup-node
---
### K8 Demo Time
- kproxy
- /etc/hosts localhost archive.org
- dashboard http://localhost:81/ui
- deployments
- scale
- secrets
- nodes
- conditions
- pods
- logs
---
### K8 Demo Time
- https://archive.org/details/camels (not located)
- https://archive.org/details/prelinger (located recently, OK)
- https://archive.org/services/whathost.php
- ```sudo docker exec -it [CONTAINER] zsh```
- ```supervisorctl status```
- ```ps fauxwww```
---
## ACT II: Transformation
---
#### Change, change, changers
- _All item changes in containers_
- only single dir visible / writable - same model:
- if item dir local:
- r/w shared volume
- process
- inside IA, item dir remote:
- rsync item over
- process
- copy back incremental / final updates (see Act III)
- if you're in Russia:
- item public?
- [YES] /compress/ pulldown
- [NO] ia jake tool!
- ia jake tool when done! (S3 back in)
---
### ... and prosper
- _workers managed by Kubernetes_
- allows ramp up/down based on webnodes
- or other processing
- could derive on primary when avail
---
#### moar
- _datanode container_
- for item serving - _readonly_
- light processing, eg:
- page/image transforms
- zipview
- ZOTF
- `/petabox/docker/supervisord.conf.datanode`
- nginx & nginx-priv
- php-cgi & php-cgi-priv
- locatord
- cron
- live-stats
- apache2
- nullmailer
- rsync
---
### Whut? (a random primary node)
| ia600600 | Container | What? | When? |
| :--------: | :-----------: | :---------------------------: | :-----------------: |
| | _datanode_ | items serving | always |
| | _workerA_ | itemA modifier, eg: fixer task| on demand |
| | workerB | itemB modifier | on demand |
| | " | " | " |
| | _worker1_ | deriving | as resources avail |
| | worker2 | deriving | as resources avail |
| | " | " | " |
| | _webnode_ | archive.org | lottery / as needed |
---
### where's the fire?
- faster OS upgrades
- decouple baseline OS and apps
- split:
- item serving (eg: S3)
- item processing (eg: EC2)
- more efficient use of IA resources
- industry std
- <redacted> dumping PHX bespoke datacenter for Google Compute Platform
- we have "< MVP" working now!
---
# ACT III: Resolution
---
### Maybes
- Could allow remote (non-IA) modifiers to upload logs w/ new/changed item files
- Container Linux / CoreOS (based on Gentoo Linux (compiled per machine, v. efficient))
- docker / rkt builtin
- super thin / small
- security paranoid; auto-updates
- auto-versions
- huge K8 contributors
- then.. [/0 .. /35] raw dirs?
- Lambda?
---
### Open Issues / Brainstorm
- logs / stats
- industry std is to send/collect to one big resource
- can drop log rotates / cron daily push/pull
- can be more realtime
- eg: logstash, rsyslog, et al
- local OCR?
- UDP locate?
- have MDAPI / redis do/handle?
---
# THE END