Tag Archives: migration

Fixing postgres

Posted on by
Reply

When upgrading my Mac, Postgres stopped working. This page documents my journey. First, the migration process gave me a long password for the postgres user. It also didn’t set up Postgres to start up automatically. netstat confirmed nothing running on 5432.

Posgres is started on the old machine using a launch daemon. I can see it at /Library/LaunchDaemons/postgresql-15.plist. Launch daemons require elevated access to run which seems related to the user recreation.

The plist file is a text file so I read it. It says that it runs at load with userName postgres. I ran dscl . list /Users and confirmed that user still exists.

I then came across this post which says I shouldn’t expect the database to start up from a time machine backup. Given that this is just a test database and there’s nothing in there I care about, I tried reinstalling Postgres. Which didn’t change anything.

I then checked /Library/PostgreSQL/15/data/log/<latest> and got:

2023-05-21 07:51:02.282 EDT [372] LOG:  listening on IPv6 address "::", port 5432
2023-05-21 07:51:02.282 EDT [372] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2023-05-21 07:51:02.282 EDT [372] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2023-05-21 07:51:02.300 EDT [402] LOG:  database system was interrupted; last known up at 2023-04-25 20:00:54 EDT
2023-05-21 07:51:02.667 EDT [402] LOG:  invalid record length at 0/331FC48: wanted 24, got 0
2023-05-21 07:51:02.667 EDT [402] LOG:  invalid primary checkpoint record
2023-05-21 07:51:02.667 EDT [402] PANIC:  could not locate a valid checkpoint record
2023-05-21 07:51:02.668 EDT [372] LOG:  startup process (PID 402) was terminated by signal 6: Abort trap: 6
2023-05-21 07:51:02.668 EDT [372] LOG:  aborting startup due to startup process failure
2023-05-21 07:51:02.668 EDT [372] LOG:  database system is shut down

That was helpful. I reset the log.

JeanneBrskysMBP:bin postgres$ ./pg_resetwal ../data
The database server was not shut down cleanly.
Resetting the write-ahead log might cause data to be lost.
If you want to proceed anyway, use -f to force reset.
JeanneBrskysMBP:bin postgres$ ./pg_resetwal -f  ../data
Write-ahead log reset

After restarting, I saw postgres up again. Yay.

nyjeanne@JeanneBrskysMBP ~ % netstat | grep 5432
222187ba86974afb stream      0      0 222187b5bb6cdf9b                0                0                0 /tmp/.s.PGSQL.5432

And the startup log is happy now.

2023-05-21 08:09:08.930 EDT [374] LOG:  database system is ready to accept connections
2023-05-21 08:14:08.987 EDT [403] LOG:  checkpoint starting: time
2023-05-21 08:14:08.992 EDT [403] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.001 s, total=0.005 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB

The other problem is that I had removed the database from pgadmin. Luckily this doesn’t delete the database. I registered it again with the same info as the settings on the old mac and all was well. My data is still there too. Nice!

converting 2K+ tests to junit 5 in one day

Posted on by
Reply

I set myself a goal to see if I could convert all the CodeRanch JForum tests to JUnit 5 in one day. And I mean with Java 5 package names, not just running them in JUnit 4 mode. (I wanted to see what edge cases I encountered). Let’s see how that went.

Forking

Ant doesn’t currently support JUnit 5. So I decided to create a fork of the code base for playing with JUnit 5. Which is fine. I wanted to automate as much of the process as possible so it’ll be easy to redo when the time comes. It’s also good because then I don’t have to worry about whether everyone’s IDE supports JUnit 5 yet. IntelliJ does if you are on the latest version and Eclipse does if you fiddle with it. Probably better to wait until after the official release of JUnit 5 and Eclipse before using JUnit 5 in the trunk anyway.

Scope reduction

I decided to only update the unit tests. We also have some functional tests with an odd hacked together way of loading the database.  (I wrote a lot of it so not criticizing others.) I’m sure this can be improved in JUnit 5. Will wait until release for that as well.

JUnit 3.8

We have 15 classes that still reference JUnit 3.8. They used a mix of a superclass that did inherit from JUnit’s test case and some that used JUnit 4 style annotations too. I decided to go through these by hand in the trunk (and my branch) and get rid of the references to JUnit 3.8. This was a pain because each one was a special snowflake. One even said it was deprecated with replacement code listed. But it took under an hour; a few minutes each. So not terrible.

As I was going, I missed noticing that the class missed annotation. I was excited to see that SonarLint flags this!

Migrating most of the code

I ran a program to migrate most of the code to the new syntax. Here’s the project on github.

This left me with 83 compiler errors to look at. Of those…

Manual clean up

  • Parameterized tests. The manual gives good examples of the options. Conversion was easy; I used a method source. I did run into two problems.
    • SonarLint false positive – If you only have parameterized tests, SonarLint still flags the class. This was fixed in Sonar per SONARJAVA-2390. However, SonarLint hasn’t been released since so need to ignore the error in my mind.
    • Eclipse false positive – If you only have parameterized tests, Eclipse doesn’t recognize it as being a JUnit test. I didn’t search to see if this was reported since it is so easy to work around. To “solve” both these problems, I added a non-parameteirzed test to the class.
  • Timeout. I had one test that used the @Test(timeout) parameter. It was clear from the manual how to convert this. I like that specifying the time uses Java 8’s Duration class. This lets you specify the timeout in a readable way rather than in milliseconds.
  • Expected exception. I had 16 tests that used the @Test(expected) parameter. I thought about automating this but decided against it. This is a good opportunity to add an assertion about the message to most of them. Which we should have done originally. But it was *so* easy to just write the type and be done with it before. Also, it isn’t 16 distinct classes. A number of them have multiple validation type methods.
  • About 25 of our tests use JMock. @RunWith(JMock.class) doesn’t go with JUnit 5. Since JMock was abandoned (last release in 2012), I decided to switch to the “long way” and call context.assertIsSatisfied(); for now. About half of them extend the same superclass at least. I also made a note that we should migrate away from JMock.

Then I attempted to run all the converted tests

I ran into one problem

  • One of our tests was relying on the order in which the @Before methods were run in the superclass. This behavior was never guaranteed and changed from JUnit 4 to JUnit 5. I fixed the code to not rely on the order. (And or course, the place this happened was a superclass test so it resulted in a large number of failures until I figured out what was going on!)

Observations

I met my challenge. 2K+ tests migrated in one day. And I observed that one superclass caused the vast majority of the special cases!

migrating speedment to java 9

Posted on by
Reply

Migrating Speedment to Java 9
Speaker: Dan Lawesson @dan_lawesson
See the list of all blog posts from the conference

Cute – Spire their mascot on github has two years experience in that role

Since a library, want to be running with Java 9 as soon as it is released.

Speedment

  • Streams API ORM – customer.stream().filter(field.equal(value)).count();
  • uses JVM memory acceleration, ode generation and modular design
  • Type safety
  • Works like streams – you don’t get any values back until terminal operation runs
  • Have non-SQL code like a collector to convert the result into JSON
  • can use findAny() with Optional on result – generates SQL limit statement
  • Have finderBy so can join tables
  • Like SQL, Streams are declared. Describe the what, not the how. But with SQL, you have to describe the result set format.

Jigsaw Effects/Problems

  • A package must only belong to one modules. Yet it is common for two jars to have same package. The first one in the classpath takes precedence. In Java 9 must have only place for package.
  • Automatic modules are for smoth transition to Java 9. It moves the Java 8 jars from the classpath to the module path. The jar automatically becomes a module
  • However, automatic modules create split pakcages and can’t have those in Java 9
  • sun.misc.Unsafe – should not be used but a key for real world Java success
  • OSGi bundling is different than Jigsaw

Jigsawing the Java 8 open source application

  • Running Java 8 under Java 9 JDK is easy
  • Created module info file
  • Brute force is to move all jars into depdencies in your monolithic module. When works, actually modularize app. Didn’t take this approach because already had OSGi modules
  • Moduler approach: create directory for each module and move relevant packages to that directory. Add empty module-info.java (no requires/exports). That won’t compile so now can incrementally add dependencies and re-compile. Since this is iterative, they wrote a script to do it.
  • Patch abuse of non-exported JDK APIs. Can add exports of java packages as a temporary workaround. Would need this flag at runtime if the temporary workaround isn’t removed. The workaround is just so you can identify all the issues and TBDs.
  • Remove the OSGI bundling. Comment it out so building a jar instead of a bundle in Maven
  • Use code generation so no reflection

Speedment Enterprise

  • Harder becuase use sun.misc.Unsafe, third party dependencies with package issues

The first 20 minutes was about the Speedment library. I felt like that was a lot for a non-product talk. I wasn’t surprised because Dan was at my lunch table. And it was interesting. It just wasn’t necessary to understand the Java 9 part. Dan made a lot of references to things earlier in the day, which was nice. Also, the path Speedment took to move to Java 9 was very useful. I would have wanted to hear more about the issues in enterprise. Are they just outstanding issues. What do they plan to do if the libraries don’t release.