Tag Archives: qcon

removing friction in the developer experience – live blogging from qcon

Posted on by
Reply

Removing Friction in the Developer Experience
Speaker: Adrian Trenaman

See the list of all blog posts from the conference

Started with a funny story explaining his talk was about removing red tape and bureaucracy. To TSA/immigration.

Goal: minimize the distance between hello world and prod. Need to be able to deploy quickly, safely and own in prod

Developer hierarchy of needs

  1. self actualize – get stuff done and have cool stories that impress your friends
  2. perks – fuzbul,beanbags,free food – we don’t work for treats. a bit like the breakfast buffet at a hotel; love at first, but then meaningless
  3. basics – laptop, wifi, vpm, eat, standing desk, screen, warmth, light

Good software org

  • Teams 3-7
  • Departments 16-24
  • Leaders not managers, leaders who code – 85% of time as lead, 60% of time as director, 15% of time beyond that
  • DevOps, ownership, open source

Work is hard – like pushing up a hill. Friction is a force that pushes back when try to do something

Friction: Staging/Testing environments

  • Too many of these such environments. Waste
  • In physical world, draw map of area and make one continuous
  • line of what need to do in order to complete job. The resulting spaghetti diagram shows wasted effort.
  • Doing this on the environment shows number of people deploying and number of deployments. Helps highlight handoffs between groups of people – dev, qa, deployers.
  • Muda – waste in process – Intellect (building environments), Overprocessing (retest in multiple enviroments), rework (environments never match prod), inventory (commits held up), transporation (deliveries to prod), motion (commit/deploy cycles), waiting (held up on someone else) and overproduction (fewer big bing releases)
  • Instead deploy directly to prod – dark canary (see if working), canary (one of X servers has new code) release (all servers get new code), rollback (if needed)
  • Think of team as a startup providing services to other dev teams
  • Teams need secure, unfettered control to their infrastructure. Break down master account into subaccounts. Also helps with cost model because can see which teams use what. Some teams need everything locked down, but not all do.

Friction: Forced technology choices

  • Voluntary adoption – let people choose technology. If successful, more will use. If nobody using, see should stop using it
  • Looks like chaos, but creating an environment where people can create own choices
  • Standards and recommendations on github: https://github.com/gilt/standards
  • Continuum of adoption by role and voluntary adoption.
  • Eventually converges on a set of norms

Friction: Fear of breaking all the things

  • Knowing going to prod makes one cautious
  • Gilt is LOSA – lots of small apps – aka “micro-frontends”. Each page considered own app
  • Gives confidence that can’t break checkout by changing the product page

Friction: Forced team choices

  • Nothing worse than working with people you don’t like
  • Leader locks down product manager, tech lead, etc.
  • Pitch and let people sign up
  • Somehow this works and everyone wants to be on the team. Everyone picks in a room on a board so can see if too many people have same skill set or too many junior people. Ultimately the tech lead chooses. Can negotitate : will do unsexy work if can also work on X. If nobody wants to work on project, think about why can’t get people excited about it. If it is operational work, can spread across teams.
  • Teams stay together 12-18 months. Better to bring work to the teams than to self-select teams every few months

Friction: Distractions

  • Coding is the primary activity
  • Everyone likes being in flow
  • Red Hot Engineer – one person is in charge of problems/distractions for a few weeks. If quiet, they can read a book or whatever
  • Minimize meetings – they have 2.75-5 hours of meetings a week. Ask at end of recurring meeting if useful and if should meet again.

Measure how doing and compare over time – delivering value, fun, ease of release, health of codebase, whether learning, missing, are we players aor pawns, speed, suitable process, support, teamwork

lessons learned from fighting nation states in cyberspace – live blogging from qcon

Posted on by
Reply

Lessons Learned from Fighting Nation States in Cyberspace
Speaker: Dmitri Alperovitch

See the list of all blog posts from the conference

Dmitri and his team uncovered 2016 DNC hack – not focus of talk because not that technically interesting
Focus on collecting a lot of data and applying AI to big data
Store data in ThreatGraph (their product) and Apollo/Hadoop

Today’s threatscape

  • Whatever business you think you’re in, you’re in the security business – hacktivists, money, etc. If have nothing of value, why in business?
  • In past, only government entities had to worry about nation state attack. Now commercial entities have to protect IP and info.
  • Examples of China stole weapons design from United States.
  • North Korea using random ware attacks – largely in South Korea – to fund weapons
  • Once you use a cyberweapon, others can use it. Ex: WannaCry is good example of reuse.
  • Inserting fake data in real data makes it hard to determine what is true.
  • Track over 40 different threat entities in China, over 10 criminal entities worldwide, 6 activitist groups worldwide, 8 in Russia and a few others around the world. Code names have animal last name – Chinese panda, criminal spiders, etc. The analyst who discovers it picks the first name.
  • Criminal actors are opportunistic. Will move on if costs too much to atack you. Nation states are more like a dog with a bone. They aren’t giving up because only one source has the information.

War stories

  • Hurricane Panda (China) – Focus on telecom for economic esponiage to benefit China.
    • webshells – web scripts to get control of webservers. They get it on the web server and then can use a browser to run any command via get requests. Typically password protect script so doesn’t return anything unless supply right password – prevents scans from finding. Attack went undetected for a year. Stole credentials and tried to remove evidence. Persisted after attack remediated.
    • Sticky keys – modify Windows registry key and then can get in without admin password. Ex: on screen keyboard runs before login. If tell Windows to run debugger first, get command prompt with full admin privilege
    • Only need a PowerShell command to steal credentials.
    • Once fixed, got thrown out in minutes. Started making typos as rushed. Continued trying to get in for four months.
    • Then they found a zero day to get admin access to machine
    • Then they finally went away and found a new victim. Dmitri’s company repeated the pattern.
    • Crowdstrike won. (article) – hackers moved on if saw CrowdStrike software on server
  • Large defense company noticed problem but couldn’t figure out how got in. CrowdStrike asked to find malware, but wasn’t one. The problem was the RSA SecurID two factor keys were compromised. Chinese thread actors stole the seeds for the token. RSA said would send seeds to company rather than storing them. However, the Chinese stole the seeds from the company directly and could VPN in using two factor.
  • Cloud VM data theft. Again no malware. Adversary had stolen API keys.
  • Other attack method to get into environment: phishing, embed powershell in a .lnk (windows shortcut files) and make .lnk file look like word doc or pdf
  • Bypassing Windows Access Control is a bunch of steps. But there is an open source tool to do all of it
  • Anti-forensic methods – delete log files, wipe data to obsfucate their activity.

Lessons learned

  • Windows is scary 🙂 [seriously though; the talk focused on Windows – presumably their expertise]. Someone asked about this and Dmitri said 95% of intrusions occur on Windows.
  • Embrace visibility/logging and AI – you will always be behind if trying to find last attack. Aggressive logging for all system help. Anonomoly based algorithms help find the unknown
  • Leverage peers – work with other entities and share information
  • Hunt for the adversary – think what you would do if you were the adversary

jvm death match – live blogging from qcon

Posted on by
Reply

JVM Death Match
Speakers:
Daniel Heidinga – IBM
Gil Tene – Azul
Thomas Wuerthinger – Oracle

See the list of all blog posts from the conference

This was a joint session of the NY Java Sig and the ACGNJ group. Fun fact – they have the URLs javasig.com and javasig.org respectively.

Graal Vision and Architecture – Thomas at Oracle

  • Java is still the primary language on the JVM but lots of others.
  • Graal compiler runs on top of JVM and can run JVM languages.
  • Truffle Framework – allows running Ruby, R and JavaScript on JVM
  • Sulong runs on top of Truffle and adds support for C/C++
  • Can mix and match languages
  • Vision: become more polyglot and more embedable

Zing – Gil at Azul

  • Only company that builds nothing but JVMs
  • Zulu is Open JDK. Open JDK only produces source code; not binaries. Zulu is 100% open source. Differntiate for embedded platforms.
  • Zing is the differentiation, namely speed.
  • Gil went over the graph about optimization that we saw in his session earlier in the day
  • Falcon is the jit compiler
  • Logic to pre-tune so runs at speed right faster

Open J9 – Daniel at IBM

  • Number 1 cloud runtime
  • In cloud, memory costs more than CPU. Three times smaller than Open JDK in benchmark
  • Have stripped down JDK so smaller image
  • Trace engine and dump engine. Free diagnostics tools – important to be able to see what JVM is doing
  • Work with hardware vendors
  • Plan to open source J( before Java 9 launches

Selection of the Q&A

  • Why use JVM? IBM said #1 cloud JVM. Azul said Open JDK for and for best tuned for Zing. Oracle said can combine with other language or compile to native code. Also Oracle disputed the performance claim.
  • How important is polyglot? Azul said have to be able to beat existing runtime to be useful. IBM said tried to create the universal bytecode and didn’t work. Oracle said performing well. Oracle said there is interest because big investment in Java source code with business logic and want to use Node.JS for small apps so can reuse. Azul said hard because people have current tool in place. I wanted to ask why this over microservices. Azul and IBM both brought up that they think that is the future. Oracle said microservices are painful over just calling the data structures. Graal allows calling Java data structures from other languages now. Azul teased him that not in prod yet.
  • R becoming more popular due to machine learning. What about speed? Oracle noted that R is very slow and interpretted so Graal helps a lot
  • What about calling C from Java? Oracle said project Panama does that. A future version of Truffle will do that.
  • Who is working on optimizing regular bytecode? Source code knows more than the bytecode does such as generics. IBM looked at but creates new problem – expolding templates – use more memory that way. Azul mourned Java 5 not going that route.
  • Javac converting lambas to a virtul call. All three panelists immedidately said the JVM can tune that.
  • Do IBM clients have prod experience with J9? Yes. Been a product for 20 years and upgraded regularly.
  • How does Oracle manage different versions? Need to pick a version of the language, not mix and match. Can use interoperability of each run in own space.
  • How does IOT affect the memory footprint? IBM said Java might not be right choice for very memory constrained environment. Beyond that, stripped down JDK could be a good choice. Azul said Zulu embedded goes into things like routers and printers. Current boundary is 11-20 MB of storage and mid-high tens of MB to run. Happy JVM can’t run light bulbs given recent hack on light bulbs. Oracle looking at what parts of JRE using and turning those parts into machine code. Does contain GC, but not many other things. Has restrictions so can’t use things like generics/reflection.
  • Do any JVMs have hard limit on memory used? Azul said yes and again teased Oracle about their product not being in production. Azul also said elastic garbage collector so kernal gets memory back as soon as GC happens. IBM has soft MX so JVM doesn’t exceed the limit for the heap. Azul noted the problem is that JVMs have dedicated padding because might need later. Providing shared padding gives this confidence – dynamically expand and shrink “insurance memory”. IBM has detection for idle resources so other processes can use that memory as headroom
  • Is Java the right language for things that appear and go away due to warm up period – serverless? Azul said it should be and working on that problem now. Even with front loading, a lot of CPU sed on startup. Working on almost instant startup but that is future. IBM saves JIT status and profile code to decrease startup time as well. Need to keep JVM around for some length of time to minimize effect of cold starts. Oracle said can produce quick start if you restrict functionality used. Moving around program beocmes less expensive compared to moving around data. Azul said don’t want to limit features. IBM said AOT is a great bandaid to solve the startup problem.
  • What happens when reach limit on number of cores? Azul disagreed with question and cited we’ve been hearing about the end of Moores Law for ages. Speed over time still increasing. Oracle said never enough so people will want more machines.