## Foms 2018
@tracey_pooh
TV, A/V, VR/AR/3D, Docker/Kubernetes
2018 Oct 15-16
_?_ for keyboard shortcuts
https://github.com/traceypooh/slides
https://traceypooh.github.io/slides/foms2018
---
# The Internet Archive
## is doing things
- AI / Machine Learning
- Third Eye
- Face-o-matic
- ARKit
- youtube-dl recording
- tvOS app
- Kubernetes _(* not really A/V but muse)_
- do _not_ talk to me about this ;-)
---
# Internet Archive
## archive.org
- WayBack Machine
- past copies of 300B+ pages
- 20M books, lendable
- 5M videos, 5M audio & live concerts
- 3M images
- 300K software items & emulation (in JS)
---
---
---
https://archive.org/tv
---
- OCR 'lower third'
- chyrons
- overlaid text on broadcasts
- not captions or descriptive text
- editorial / summarizing in nature
- 4 TV channels, 24x7, ~1 min from realtime
- CNN
- MSNBC
- Fox News
- BBC News
---
# Chyron filtering
- tesseract OCR
- free; errors
- simhash
- groups 'nearly the same'
- character flips
- word off in time
- look for vowels
- pick 'most seen' group every minute
- and tweet
---
AFTER WH MEETING, SCHUMER DISHES
WHEN HE THOUGHT NIC WAS OFF
---
# bots
- twitter bots
- https://twitter.com/tvThirdEye
- https://twitter.com/tvThirdEyeB
- https://twitter.com/tvThirdEyeF
- https://twitter.com/tvThirdEyeM
- https://twitter.com/tvThirdEye/lists/all
---
https://tweetdeck.com
---
# API
- https://archive.org/details/third-eye
- TSV
- cropped imagery
- raw + filtered OCR
---
# Research
- How cable news networks covered Kavanaugh-Ford hearing
- Aaron Williams, Danielle Rindler, Chris Alcantara,
- Kevin Schaul & Kevin Uhrmacher
- https://www.washingtonpost.com/graphics/2018/politics/kavanaugh-ford-hearing-chyrons/
---
# Research
- audio fingerprints
- presented keynote paper on CSPAN floor speeches and vocal pitch Bryce Dietrich, UIowa
- discovered 375K political Ads
- find sound bites of speeches
---
# CNNs
- Convolutional Neural Network
- filtered neural network
- each layer uses output from prior layer as input
- multi-node connections (but not "fully connected")
---
# CNN Example
- feed in image
- node looking for eyelash
- node looking for iris
- could feed to node looking for eye
- meanwhile... nose node
- all feed to face recognizer node
- could feed to "is this Barack Obama?"
---
# Face-o-matic
https://blog.archive.org/2017/07/19/introducing-face-o-matic/
---
# ARKit / OpenFace
- implementation of FaceNet
- https://cmusatyalab.github.io/openface/demo-3-classifier
- similar to tensorflow (Torch..)
---
# Image Matching
- pixel diff algorithms (MAE, RMSE, MSE)
- perceptual hashing pHash.org
- image => _8x8 grayscale_
- convolve to 8x8 image with DCT
- reduce to _64bit_ number
- hamming distance Int64 pairs
---
### pHash - to gray 8x8
---
# OpenFace Training
- 3+ images per person/face
- avoid 'overfit'
- align eyes + nose (nostrils?)
---
### miniARchive
---
## bonus round
---
## GitLab + Auto DevOps + Kubernetes + Docker
https://archive.org/details/auto-devops
---
## appleTV TV app
https://github.com/traceypooh/TVArchive
tvOS + TVJS + TVML is cool!
---
# Webamp
## Don't click the llama!
https://archive.org/details/otr_cbsradiomysterytheater
---
# The End
---
### miniARchive
---
## bonus round
---
## GitLab + Auto DevOps + Kubernetes + Docker
- OCR 'lower third'
- chyrons
- overlaid text on broadcasts
- not captions or descriptive text
- editorial / summarizing in nature
- 4 TV channels, 24x7, ~1 min from realtime
- CNN
- MSNBC
- Fox News
- BBC News
---
# Chyron filtering
- tesseract OCR
- free; errors
- simhash
- groups 'nearly the same'
- character flips
- word off in time
- look for vowels
- pick 'most seen' group every minute
- and tweet
---
https://tweetdeck.com
---
# API
- https://archive.org/details/third-eye
- TSV
- cropped imagery
- raw + filtered OCR
---
# Research
- How cable news networks covered Kavanaugh-Ford hearing
- Aaron Williams, Danielle Rindler, Chris Alcantara,
- Kevin Schaul & Kevin Uhrmacher
- https://www.washingtonpost.com/graphics/2018/politics/kavanaugh-ford-hearing-chyrons/
---
# Research
- audio fingerprints
- presented keynote paper on
---
# ARKit / OpenFace
- implementation of FaceNet
- https://cmusatyalab.github.io/openface/demo-3-classifier
- similar to tensorflow (Torch..)
---
# Image Matching
- pixel diff algorithms (MAE, RMSE, MSE)
- perceptual hashing pHash.org
- image => _8x8 grayscale_
- convolve to 8x8 image with DCT
- reduce to _64bit_ number
- hamming distance Int64 pairs
---
### pHash - to gray 8x8
https://archive.org/details/auto-devops
---
## appleTV TV app
https://github.com/traceypooh/TVArchive
tvOS + TVJS + TVML is cool!
---
# Webamp
## Don't click the llama!
https://archive.org/details/otr_cbsradiomysterytheater
---
# The End