[QCon 2019] Low Latency in the Cloud, with OSS

Mark Price @epickrram

For other QCon blog posts, see QCon live blog table of contents

Requirements

  • Trading app
  • Need microsecond (not millisecond) response time
  • Need data in memory vs database
  • Lock free programming
  • Redundancy
  • High volume
  • Predictable latency

Hydra

  • System built on OSS
  • Opinionated framework to accelerate app dev
  • Clients communicate with stateless, scalable gateways
  • Persistors – manage data in memory.
  • Gateway – converts large text message to something smaller and more efficient

Design choices

  • Replay logs to reapply changes. Business logic must be fully deterministic. Bounded recovery times
  • Placement group in cloud – machines guaranteed to be near each other. Minimizes latency between nodes

Testing latency

  • Do as part of CD pipeline
  • Can’t physically monitor with fibertab
  • Capture in histogram to get statistical view and calculate data
  • Test under load
  • Fan out where test from
  • Store % in time series data
  • Can see jigger for garbage collection

Performance on shared box/cloud

  • Not in control of resources running on
  • Containers share L3 cache so can see higher rates of cache miss
  • CPU throttling effects
  • Hard to measure since can’t see what neighbors are doing
  • One option is to rent the largest box possible and compare to vendor website for specs. If have max # cores, know have box to self. Expensive. Was about five dollars a year. At that price, might be worth just buying own machine in data center
  • Can pack non latency services onto shared machines

<missed some at the end. I got an email that distracted me>

My impressions

There was a lot of discussion about the histogram. I would have liked to see some examples rather than just talking about how it is calculated. They didn’t have to be real examples to be useful. There were some interesting facts and it was a good case study so I’m glad I went. I was glad he addressed that non-cloud is a possible option for this scenario

[QCon 2019] Making npm install safe

Kate SIlls @kate_sills

For other QCon blog posts, see QCon live blog table of contents

General

  • Building financial software in JavaScript
  • 97% of code in a modern web app comes from npm

Security issues

  • All packages are risky
  • Imports and global variables
  • Effects opaque
  • Can be from dependency many levels deep

Pattern

  • Event stream package (11/28/18)
  • Electron native notify package (6/4/19)
  • Can call node built it modules to read a file and send it over the network
  • Targetted cryptocurrency

Options for solution

  • Write everything yourself – not scalable
  • Pay open source maintainers so someone responsible for security – people make mistakes. Even people who are paid can compromise a system
  • Code audits – don’t see everything, Hard to find clever 

Other approach

  • Preventing attacks requires infallability
  • Better to look for ways to limit damage
  • For example, would be better of if can’t import fs
  • JavaScript is good at code isolation. Clear separation between pure computation and connection to outside world

Realms – draft proposal

  • Want to be able to create realm without overhead of an iframe
  • Featherweight compartment – shares primordials/context
  • There is a realm shim now
  • Self/window not defined in the compartment

Attack – prototype poisoning

  • Save copy of original function
  • Do something bad first and then call original function so it looks right
  • SES (Secure ECMAScript)  – realms + transitive freezing/hardening
  • Can’t change prototype behavior with SES
  • npm install ses
  • SES.makeSESRootRealm()

POLA

  • Principle of least authority
  • Same as principle of least privilege
  • Reasonable to want to access file system. Can attenuate (reduce the impact of) access by wrapping fs with check for correct file name. (Not clear how prevents using original fs). Method harden protects
  • The chalk package needs process/OS access to change color
  • But can kill process and change priority of process with that access
  • Want to limit access to just what needed
  • Chalk only needs OS to get the release. Can attenuate so just have that one function to return release string.
  • Object capabilities – http://habitatchronicles.com/2017/05/what-are-capabilities/

Moddable XS

  • Only completed ECMA Script 2018 engine optimized for embedded device
  • Contains SES
  • Safe for users to install JS apps
  • Can only do specific things
  • Can add own app to washing machine

Metamask’s Sesify

  • Ethereum wallet
  • Can run Ethereum apps in browser without running full Ethereum node

Salesforce’s Locker Service

  • One of primary co-authors of Realms and SES
  • Plugin platform

Caveats for Realms

  • Work in progress
  • Have to stringify to use
  • Still in draft

Q&A

  • What if add something bad? https://ocapjs.org/t/tofu-trusted-on-first-use-tool/27 Putting something bad in wrapper would show up in diff/code review.
  • How SES different than Object.freeze? Object.freeze only freezes that instance and doesn’t go up prototype chain 
  • How know what functions/authorities need to provide to packages? Still developing patterns of use. For now might be trial and error. Might need changes to module.
  • Why don’t we hear about npm install attacks in other languages? Still have problems. Java can’t protect [I raised my hand and described how Sonatype helps protect Maven Central]. Worse on JavaScript because lots of tiny packages. Visibility will help in future.
  • Will this be bolted on to web frameworks? Hasn’t yet, but hope will happen.

My impressions

While I was aware of the problem, the solution (or future solution) is really interesting! She left lots of time for Q&A which was nice after yesterday. [My track didn’t have much time for Q&A in most sessions]

[QCon 2019] Are we really cloud native?

Bert Ertman

For other QCon blog posts, see QCon live blog table of contents

Cloud Computing

  • Not new
  • Market growing fast/analysts on rise
  • “Java EE is dead, long live the Cloud” – cloud coming at expensive of Spring, etc
  • “There is no cloud. it’s just someone else’s computer” – 5 years ago was just virtualization elsewhere. No longer does it justice
  • Evolution – IaaS -> PaaS -> Serverless
  • Serverless is the evolution of virtualization or compute
  • Re-imagine middleware or higher level services as managed services that can call via an API
  • Cloud native is the step after serverless

Mapping

  • Business agility – Microservices
  • Infrastructure = CI/CD + containers
  • Process = Agile + DevOps

Evolution and problems

  • 80-90% of IT budgets are spent on maintaining existing systems
  • Experiment with new tech/process comes out of time left
  • Don’t save money by simply moving the app server to the cloud. Often costs more.
  • Then tried spring boot with a fat jar which turned into an inverted app server
  • Adding Docker makes it more portable but doesn’t actually use benefits of cloud
  • Next tried microservices in Docker. Waste more resources because need more virtual machines. Introducing problems while solving other problems. Modularity is good and microservices are a modularity tool. However adding cost due to network/config/dependencies/versioning/etc
  • Next tried Kubernetes. Everyone shouldn’t have to run/manage in prod
  • Agile adoption took a few years because needed business buy in. DevOps isn’t just learning tools. DINO (devops in name only)
  • Cloud native is a dev ops journey. Continuous journey with new services and components. Services can be short lived. Think about managing a mix of software and infrastructure and scale
  • Get to a mix of serverless and non-serverless services.
  • Technologies or frameworks are not cloud-native, it is the way you use them

Other Benefits

  • Economic disruption – startup costs low. Don’t need datacenter staff
  • Easily experiment with new tech or new business ideas
  • Faster time to market

Tips or challenges

  • Use managed services where possible
  • IT is not just a cost center; need strategy
  • Business needs to trust IT

Java

  • GraalVM and compiling to native code facilitates writing serverless/lambda. Solves cold start problem
  • If Java is your only skill, you are in for a hard time
  • With DevOps, there are new problems you need to be knowledgeable
  • Cloud Engineer needs to know more than just a programming language. Flowchart: https://github.com/kamranahmedse/developer-roadmap/blob/master/readme.md

Q&A

  • OSS advice? OSS Community bundling products to help with direction of cloud native. Try to use provider supplied services where possible.
  • Stats on whether spend less in serverless? Maybe. Definitely war stories from real enterprises

My impressions

Bert got a lot of laughs which is good. It means the audience is engaged. It’s a good perspective and I like the path/journey he took to get there.