Todd Montgomery, Kirk Pepperdine and Bert Ertman

For other QCon blog posts, see QCon live blog table of contents

Quotes

• “If you don’t create garbage, you aren’t going to have a problem with the garbage collector” – Kirk
• ”Performance is in the eye of the beholder” – Bert
• “Once you see it you can’t stop seeing it” – Kirk
• ”Need to adjust containers in a sensible way” – Kirk

GC

• Memory churn is an issue. But observability bad so cant see. -Kirk
• GC allocation rate not the problem. It’s the result of the collection – Todd. Immediately Kirk disagrees
• Don’t fear allocating objects, but don’t be wasteful. Parsing a String in the JDK is wasteful – Todd
• Allocation rates still matter. Have to use CPU to allocate. Can fill a water tower one spoon at a time -Kirk

Solutions

• Write simple code first -Todd
• Never stop making it better. You may pause or decide not enough time. But never really done – Todd [this sounds like good enough]
• A good engineer wants it to take less time. Will plan up front to type less. But also can’t stop thinking about how to make it better. However, might not be able to get the chance to do so. -Todd. As a counter, Bert noted that time runs out eventually and ok i have met no-functional requirements.
• Get data – Kirk

General

• First question is whether what customer wants is possible – Todd
• A lot of the tools lie to you – Kirk
• Profile disruption – Kirk
• Have ore than one tool – Todd

Tools

• Async profler – Todd
• Flight recorder Todd. Looking at adding perf data – Kirk.
• perf on linux – Todd
• Remove/reduce logging – Todd. Seconded by Kirk. Good at finding the problems that have already been solved and likely won’t occur again.
• perf – Kirk
• GC logging – Kirk
• Jmeter and other tools. Use with cloud based apps – Bert

Cloud vs hardware

• Can start taking some variables out for serverless – Todd
• There’s a lot between you and the actual hardware. Some tricks don’t work -Todd
• Idiomatic parts of Java still work. Simple designs. Decoupled. No shared state – Todd
• Underprovisioning other parts of hardware. Ex: focus on CPU and not network – Kirk
• JIT compiler can produce different results depending on part of memory used – Kirk
• Can look at assembly generated from JIT to counter this – Todd
• Serverless often used as glorified REST API or glue code More config needed- Bert. Kirk countered that won’t happen because IT compiler is still going half an hour For complex code, could be hours or never for JIT to have stable plan. Sometimes stabilize in less than optimal config. Emergent just about every time restart – Kirk

Questions

• Spring boot? Not using would be premature optiimization. Can tweak after – Kirk
• How much knowledge of OS should have? As much as can -Kirk. It’s something that is there and have to live with. The more you understand it, the less it seems like magic. – Todd, The better you know environment the better can accoomdate it – Kirk. Better understand illusions. Writing to memory not immediate – Todd. Kirk asked who knows what a DDR4 chip layout is. More than zero :). Incorporated into chip design
• Performance tuning on how deal with which quadrant? Yes. Deal with threads being on different cores with different profiles. Can only control so much. Don’t take into account unless want to go fully into it – Todd. Unless you are Todd, don’t go off heap – Kirk
• Performance difference between Open JDK and Oracle JDK? Many JDKs based on same Open JDK core Difference in patches applied for bugs backported So performance shouldn’t vary. -Kirk Everyone should be supporting Adopt Open JDK. Only community supported distribution – Todd and Kirk. Todd uses Oracle JDK less but his clients do.
• Cloud JDK? Depends on distribution. Amazon has own JDK now. Also bring your own runtime – Bert
• More on how profilers lie? Not malicious. Profilers sample. A lot happens between savepoint – Todd. One thread can stop all others including GC. Samplers can show conflicting views of the same data. Biggest gain is o get application do wha wanted it to do in the first place. A lot of profilers work on differentials – Kirk
• How generate less garbage? Find hot allocation sites and get rid of them -Kirk. Primitive data structures using boxed version instead of actual primitive version. Consider primitive instead of Object Be careful, but sometimes allocating a pool helps. Not always the best solution. Look at API have Can you reuse common aspects – Todd
• Avoiding mutable state increases allocation rates. Kirk cringed. Doesn’t like mutable state, but not going around making everything immutable. Final comes with consequences. Proper encapsulation helps avoid need for immutability. Do you real need getter/setter Embed responsibility in object instead – Kirk. Absolutes are wrong. There’s always exceptions. Mutable state not a bad idea. Shared mutable state is the problem. Have one thread handle writes. – Todd. HashMap not a word used in business terminology. Missing a domain term in model – Kirk. Indiscriminate use of final is a bandaid over modeling issue – Kirk
• How avoid getters in CRUD apps. Need gets at edge of system. Ex: database, GUI. In middle of system, don’t need to externalize – Kirk, Streaming set of change – Todd
• Microservices? Not all services are designed for reuse. Time to market is the driver – Bert. REST is terrible from a model perspective. A lot of time spent parsing ASCII – Todd. Can parse a lot of ASCII in one network call – Kirk
• Serialization? Dislike ORM as already had data. Easier to learn SQL – Todd. Kirk disagrees with SQL part. Wire friendly formats better than serialization. Or make them events – Kirk
• When god to use object pooling? Someone said never. (I thin Kirk) Needs to pay for itself – Todd. Small object that are churned will be removed by the JIT. Large objects that are expensive may pay or self. But then it is like writing own GC. – Kirk
• What features most excited for? Don’t look what coming to avoid disappointment – Todd. Fibers. Adding more work generally makes things slower. But fibers reduce thread count so might have benefits in some situations – Kirk
• How make custom profiler? Look at problem to see what info need. Extend what need. Could be bytecode analyzer or an existing profiler. No two are the same when customize profiler. Tune in prod. Profilers will like sometimes. Test environments almost always lie. So lower sampling rate/impact since run in prod – Kirk. Very few organizations have test environment that mirror prod in any real way. Happy accident when tuning test environment improves prod performance – Todd
• What resources recommend? Mechanical sympathy mailing list. – Kirk

My impressions

The three panelists chatted for a while before the panel started. They clearly look just as comfortable on stage then when just standing around. They interact well ad it is fun. It would have been nice if they had more than one mic to share across the three of them. During the day, I understand there re only so many mics. But right now only two rooms are in use. So I wish we could have been given the Q&A mics to borrow from other rooms. I learned a lot. Also, I brought a snack in case was hungry since the session starts/ends late in the day. I was engaged enough by the session that I wasn’t hungry enough to be distracted! Also Sai: thank you for volunteering to run the mic!