Transcription

Building a security appliancebased on FreeBSDBSDCanOttawa 20191

Mariusz ttawa 20192

3

Dataencryption4

DataencryptionStorage5

ExternalstorageDataencryptionStorage6

RemoteaccessExternalstorageDataencryptionStorage7

yptionStorage8

Data Encryption9

Data Encryption GBDE GELI native ZFS encryption10

GBDE - Geom Based Disk Encryption FreeBSD 5.0 AES-CBC 128bits Different key for each write CPU overhead disk space overhead11

GELI Many cryptographic algorithms AES-XTS AES-CBC Blowfish-CBC Camellia-CBC 3DES-CBC Integrity verification (HMAC) Don’t have such overheads like GDBE One-time key12

Keeping encryption keyAppliance: Use memstick Need only during boot Initialize during first bootVM: Use passphrase Use no encryption13

Storage14

Storage ZFS UFS15

ZFS checksums snapshots compression RAIDZ16

ZFS - checksum fletcher2if (id 0 id channels alloc)if (id 0 id channels alloc) fletcher4jle 30jl 3039 45 08 7c 1a 8b 4539 45 08 7e 1a 8b 450111110001111110 sha256 sha512 skein17

ZFS - compression GZIP lz4 ZSTD# zfs list -o r/crashlz411.17x18

ZFS - compression GZIP lz4 ZSTD# zfs list -o r/crashlz411.17xProblem: What if customer want to backup the data?19

ZFS - snapshotsA snapshot is a read-only copy of a file system or volume.Snapshots can be created almost instantly, and they initiallyconsume o additional disk space within the pool. However, as datawithin the active dataset changes, the snapshot consumes diskspace by continuing to reference the old data, thus preventing thedisk space from being freed.https://docs.oracle.com/cd/E23824 01/html/821-1448/gbciq.html20

Snapshots - cluster multi-masterMaster0New DatacontinuousreplicationMaster1New Data21

Snapshots - cluster multi-masterMaster0New DatacontinuousreplicationMaster1llatosseccaatadNew Data22

Snapshots - cluster multi-master# zfs a/data/12345678/dumps/data/local/dumps23

Snapshots - cluster multi-master# zfs list -t snapshotNAMEUSEDAVAILREFER data/data/local/[email protected]

ZFS sending & receiving snapshotsBefore r317414:After r317414: ZFS decompress FS to send ZFS FS send over SSH manual compress FS to reduce ZFS FS receive over SSHlatency sending over SSH manual decompress FS receivedover SSH ZFS compress FS which wasreceived25

Downside of using ZFS snapshots Data loss after rollback Can’t rollback ZFS changes Snapshots can take a lot of space on clustermulti-master26

Downside of using ZFS snapshots Data loss after rollback stnioCan’t rollbackZFS changespkcehcs Snapshots can takekr a lot of space on clusteramkomulti-masterob27

Downsides of ZFS Not enough RAM to import pool No full disk encryption If something will go very wrong we still want tobe able to do something What about factory reset?28

Read only file system - UFS GELI&ZFS for customer data Contains read-only operating system Data are not encrypted If something goes wrong we can still boot from it Try to reflect some ZFS promises29

Read only file system - UFS# gpart show -l ada0 stem0-0[bootme] 408116793067276data0(5.4T)(64K)30

RAIDZ231

Reflect RAIDZ2 with UFSNamemirror/system0StatusComponentsCOMPLETE gpt/system0-0 (ACTIVE)gpt/system0-1 (ACTIVE)gpt/system0-2 (ACTIVE)gpt/system0-3 (ACTIVE)gpt/system0-4 (ACTIVE)gpt/system0-5 (ACTIVE)32

Reflect RAIDZ2 with SWAPNamemirror/swap0StatusComponentsCOMPLETE gpt/swap1 (ACTIVE)gpt/swap2 (ACTIVE)gpt/swap0 (ACTIVE)mirror/swap1COMPLETE gpt/swap3 (ACTIVE)gpt/swap4 (ACTIVE)gpt/swap5 (ACTIVE)33

Upgrade steps34

Upgrade steps - Boot from system0# gpart show -l ada0 4011721045101ada04012816883886082system0-0[bootme] 408116793067276data0(5.4T)1GPTboot0(5.5T)(64K)35

Upgrade steps - override system1 and set bootonce# gpart show -l ada0 4011721045101ada04012816883886082system0-0[bootme] (4.0G)838877683886083system1-0[bootonce, bootme] boot0(5.5T)(64K)36

Upgrade steps - reboot37

Upgrade steps - bootloader removes bootme# gpart show -l ada0 stem0-0[bootme] (4.0G)838877683886083system1-0[bootonce] )38

Upgrade steps Create zfs snapshot Upgrade error accrued Reboot machine39

Upgrade steps - boot from partition with bootme# gpart show -l ada0 stem0-0[bootme] (4.0G)838877683886083system1-0[bootonce] )40

Upgrade steps - rollback# zfs rollback -R [email protected]# gpart show -l ada0 stem0-0[bootme] (4.0G)838877683886083system1-0[bootfailed] )41

Upgrade steps - upgrade succeeded# gpart show -l ada0 stem0-0[bootme] (4.0G)838877683886083system1-0[bootonce] )42

Upgrade steps - upgrade succeeded# gpart show -l ada0 stem0-0(4.0G)838877683886083system1-0[bootme] )43

Hot plugnotify 20 {match "system" "DEVFS";match "type" "CREATE";match "cdev" " ada[0-9] ";action "/usr/local/bin/newdisk cdev";};44

External Storage45

External storage NFS iscsi SAN over FC46

External storage - NFS NFS No encryption No authorization iscsi Is it corporate solution? Able to mount on multiple machines SAN over FC47

External storage - iscsi NFS iscsi SAN over FC Encryption Authorization Is it corporate solution? Not able to mount on multiplemachines48

External storage - SAN over FC NFS iscsi SAN over FC Encryption Authorization It is a corporate solution Not able to mount on multiplemachines49

External storage - SAN over FC NFS iscsi SAN over FC with GELI Encryption Authorization It is a corporate solution Not able to mount on multiplemachines50

Redundancy Use at least two FC cards Combine multiple luns with gmultipath51

Remote access52

Access the box Through SSH We don’t want to customize our builds perclient In theory we can have an key per client SSH keys Hard to hijack What if your engineer change the job? We have to be in customer network53

Access the box - exotic IPMI Some video conference (like webex) No SSH keys So maybe password after all? But password is easy to hijack What if yours enginner change the job?54

Implementing S/Key (whlkey)55

Implementing S/Key (whlkey) We configure it as: 50 keys per day The key length is 16 chars Key is rotated every day Unified password: O 0, I l, etc. The secret is stored in some trusted machine The engineer can only get keys for this week56

Process security57

Basic problem You can’t build everything from scratch You can’t audit everything Who you really trust?58

Basic problem You can’t build everything from scratch You can’t audit everything Who you really trust?Security stops where the trustbegins59

Privileged separationPrivilegedSIMPLE IPCUnprivileged Reduce TCB Simple communication60

Privileged process Have access to: DB Storage Network Authenticate unprivileged process Extend capabilities of unprivileged process61

Unprivileged process Have access to storage by single FD Have access to network by single/two FD Implements complicate logic Is sending a simple commands asking privileged process Limited RAM Limited CPU time62

Privileged separation with FreeBSDCapcisumPrivilegedlibnvUnprivileged63

Capsicum tight sandboxing (cap enter(2)) capability rights (cap rights limit(2))64

Libnv nvlist create Types: nvlist add {type} string nvlist get {type} number nvlist take {type} bool nvlist move {type} nvlist nvlist send descriptor nvlist recv binary nvlist destoy array65

Privileged separation - is it PFreeTDSfreetypelibX1166

Privileged separation - network daemon Privileged process iswaiting for connection67

Privileged separation - network daemon Privileged process isClientwaiting for connection New connection fromclient68

Privileged separation - network daemon Privileged process isClientwaiting for connection New connection fromclient Fork and createunprivileged process69

Privileged separation - network daemon Privileged process isClientwaiting for connection New connection fromcredentialscredentialsclient Fork and createunprivileged process Client is authenticating70

Privileged separation - network daemon New connection fromclientClient Fork and createunprivileged processRamCPU Client is authenticating Privilegiat process israising unprivilegedprocess limits71

Privileged separation - network daemon Fork and createClientunprivileged process Client is authenticating Privilegiat process israising unprivilegedprocess limits Creating connection tothe serverServer72

Privileged separation - network daemon Client is authenticatingClient Privilegiat process israising unprivilegedprocess limits Creating connection tothe server Pass connection tounprivileged processServer73

Privileged separation - network daemon Client is authenticatingClient Privilegiat process israising unprivilegedprocess limits Creating connection tothe server Pass connection tounprivileged processServer74

Privileged separation - network daemon Privilegiat process isClientraising unprivilegedprocess limits Creating connection tothe server Pass connection tounprivileged process Create a dump fileDump fileServer75

Privileged separation - network daemon Privilegiat process isClientraising unprivilegedprocess limits Creating connection tothe server Pass connection tounprivileged process Create a dump fileDump fileServer Pass dump file76

Privileged separation - network daemon Privilegiat process israising unprivilegedClientprocess limits Creating connection tothe serverDump file Pass connection tounprivileged process Create a dump fileServer Pass dump file77

Other methods Jails CloudABI78

Thank you!Mariusz ttawa 201979

Building a security appliance based on FreeBSD BSDCan Ottawa 2019 1. Mariusz Zaborski 2 [email protected] [email protected] https://oshogbo.vexillium.org @oshogbovx BSDCan Ottawa 2019. 3. 4 Data encryption. 5 Data encryption Storage. 6 Data encryption Storage External storage. 7 Data encryption Storage External storage Remote access .