Getting your CI/CD pipeline right can be a daunting task. Here is one I had to address:

• Create a Quarkus Java application
• Compile it to a native executable
• Build a container for it
• Make the container available to both Linux and MacOS

The little irony, Docker on macOS or Windowsruns Linux under the hood.

The easy part - Quarkus

As I've written before it is easy to get started with Quarkus. It provides 5 ways to build containers, and detailed instructions to build a native image.

Building a native image looked daunting, with quite some prerequisites like GraalVM, CLI and C compiler. Luckily all this is available in a builder image, and a simple property settin in your pom.xml settles it:

I'm stronlgy in favour of Contract-First Development, when in comes to APIs. All invested parties, including your future self, agree on a neutral format, that both API providers and consumers will stick to. For REST APIs that is the OpenAPI spec

A popular critique of that approach is that it reeks of Big Design Upfront, happily skipping over the fact that nothing stops the teams to iterate over the specification too, one path, one schema at the time

The source of truth

The specification becomes the single authorative source for endpoints, security requirements, data formats and responses. While it is possible to generate the spec from source code, like. e.g. Spring or Quarkus, I see clear advantages to provide the specification standalone. Create it with a tool like Apicur.io, Stoplight or APIGit. Or use a plugin (or another) in your IDE or the other one

Once you have your first draft, you want to implement it server side. Eclipse vert.x offers the Vert.x OpenAPI Router for exactly that. You can get more details from my OpenAPI talk, or by peeking into the sample project (which used vert.x inside Quarkus).

When end-2-end testing applications that use an IdP, an IdP needs to be in a known state to make test repeatable.

Typically a container is used, with a configuration that needs to be reset before (and after) a run. Restoring the IdP configuration isn't ideal, since addring new test cases (e.g. adding a user with different properties to check application behavior). I propose a different approach: One-off IdP

Container without persistence

I start with an empty deployment of KeyCloak running in a docker container.

#!/bin/bash
#Run a clean KeyCloak
docker run --rm -p 8080:8080 \
       --name testcloak \
       -e KEYCLOAK_ADMIN=admin \
       -e KEYCLOAK_ADMIN_PASSWORD=password \
       quay.io/keycloak/keycloak:latest start-dev

The --rm parameter ensures that the container is discarded after use. There is no persistence flag (--mount), so when the container goes down, all data perishes (and that's intendet).

Configuration sequence

The empty KeyCloak only knows the realm master and the user admin. To turn it into a fully functional IdP we need to configure it. Since we want this process to be repeatable we shall use Keycloak's REST API. The documentation is complete, including an OpenAPI spec, but in a dictionary style, so all is good when you know what you are looking for. To learn what is needed the browser development tools while using the admin UI teach us the what.

The Domino REST API delivers collections using chunked transfer encoding. This has the advantage, that you can process results as they arrive. It produces the challenge that the usual client side code is designed to first wait for completion of the request. I wrote about the JavaScript solution a while ago, this is the Java edition.

Client choices

In JavaScript land the choice of client is simple: the Fetch API. In Java we have some choices:

• Since Java 11 there's a HttpClient in the JDK
• A very popular library is provided by the Apache HTTP components
• The SPRING framework, undisputed king of Java web frameworks, offers its own sset of clients
• The vert.x framework, which we used to build DRAPI, features its own client

There are probably more around. This article uses the JDK HttpClient. I'll skip the parts with Authentication and TLS handling, check the full example for details.

How it works

First we create an java.net.http.HttpClient. It takes care of the http version and the TLS context.

HttpClient getClient(SSLContext sslContext) {
  return HttpClient.newBuilder()
           .sslContext(sslContext)
           .build();
}

Then we build and execute the request. The magic is the BodySubscriber (more on that below).

Integer runGetRequest(HttpClient client, String url, String authHeader, BodySubscriber subscriber) throws Exception {
  HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Authorization", authHeader)
            .GET()
            .build();

  CompletableFuture<Integer> response =
          client.sendAsync(request, responseInfo -> subscriber)
          .whenComplete((r, t) -> System.out.println("Response: " + r.statusCode()))
          .thenApply(HttpResponse::body);

  return response.get();
}

You are developing a web application using Quarkus that consists of multiple (micro)services and a bunch of supporting libraries. Since maven modules provide sufficient isolation, you decide to use a parent project to keep dependency versions and parameters in sync and a Maven Reactor to build them together.

This blog post is for you. A special thanks to Alexey for helping out.

Moving parts

Our objective is to have a development setup where we can edit any of the services or libraries and then run them individually or all together. Ideally without the need to alter configurations between runs and the ability to deploy the setup using devcontainers (note: that's about the development setup, not about deploying the finished application). There are some moving parts:

• Quarkus CLI or Maven plugin
• Multi-Module Maven to keep libraries and services together and apart
• Maven profiles to conditionally modify maven runs
• Maven's <pluginManagement> and <dependencyManagement> to define configurations without activating them
• Your favourite Ide configured for Java and Quarkus
• Understand how VSCode's Tasks work
• Know how to setup DevContainers, especially when you need sidecars

That's a lot, let's dig in.

In the beginning was a test, that provided to be vital, but not sufficient. In modern application development we encounter:

• Unit Tests: testing a single function or a tuple of related functions
• Module Tests: testing bigger parts of your application, without actual external dependencies
• Integration or End-to-End tests: a.k.a life firing exercise

This entry isn't a discussion about the merits of how much and when test, but making tests easy to setup and distinguish

The manual way

We typically use Mockito, vert.x and REST-assured in our tests, so a typical test class would look like this:

@ExtendWith(VertxExtension.class, MockitoExtension.class, MyCustomExtension.class})
@Tag("UnitTest")
class SomethingTest {

  @Test
  does_it_blend() {
    // Test goes here
  }
}

It is just two lines, but everywhere. You can simplify it by creating your own annotation.

The custom annotation

@Target({TYPE, METHOD, ANNOTATION_TYPE})
@Retention(RUNTIME)
@ExtendWith({VertxExtension.class, MockitoExtension.class, MyCustomExtension.class})
@Tag("UnitTest")
public @interface UnitTest {
  // no action needed here, JUnit use only!
}

Now you simply use:

@UnitTest
class SomethingTest {

  @Test
  does_it_blend() {
    // Test goes here
  }
}

While this looks like minor cosmetic, it allows to control test extensions from a single place, your annotation source. Repeat that process for the other test types (ModuleTests, IntegrationTests, PerformanceTests etc.) you want to use.

In your pom.xml, in the build-plugins section you can use the tag to ensure all your unit test, but only them execute on mvn test and the others on mvn verify

<plugin>
  <!-- Run UNIT and MODULE tests, no backend calls -->
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <groups>UnitTest,ModuleTest</groups>
  </configuration>
</plugin>

As usual YMMV

Know your patterns, so you don't re-invent the wheel. Patterns are like LEGO: You can combine them to larger patterns. One of those is the Hexagonal Achitecture officially called:

Ports and Adapters

proposed by Alistair Cockburn, one of the co-signers of the Agile Manifesto.

I've written about his work before and can highly recommend his latest book "Hexagonal Architecture Explained" (or for that matter any of his writings).

In a nushell, Ports and Adapters proposes an approach that leads to loosely coupled systems, that separates business logic from the software environment (think I/O, databses, API and UI) to facilitate clarity and ease of testing.

This post doesn't aim to explain it, for that you can refer to the Wikipedia article or, better, read the book but trying to describe how to apply it to Java in a Maven project.

Accessing applications usually entails some kind of identity. Some part(s) of your application provide identity (called IdP), while other's consume it (paraphrased from Captain Obvious). Identity could be provided from a record or document in your or another database, an LDAP directory, an OICD or a 3d party like your eMail provider or social account, or with some hoops and loops Webauthn (a.k.a passkey).

The question is: how deep does it go?

A typical archtecture

For simplicity I'm skipping load balancers and cache facilities

Each tier might or might not have its own identity, lets have a closer look

(0) User tier

When is authentication information stored at the user tier (think Browser, app or OS) and how immediate is it? Like a username/password or a passkey private key. Those can be exchanged for a token. Or is it a session cookie or an access_token allowing immediate access? How are those credentials protected and/or synced? How vulnerable are they on physical access?

(1) Access tier

A.k.a the firewall or VPN Does it assert a valid user? Are the credentials the same or different from your application. Does your access tier include IP ranges or geo location as part of identity? I've seen identity requirements 100% on VPN, but rarely on firewalls

(2) Web tier

TYpically you find a static web server like nginx, Apache http, the Kubernetes Ingress or a service by a cloud provider.

Even when all your static resources are served by your application tier, you can identify your web tier where requests flow through. When you can establish identity there (and reject invalid ones), you have one more protective layer. nginx+ can do that with JWT

(3) Application tier

YOur application could be a monolith, microlith, microservice, follow a layered architecture, be message driven, event driven or be contemporary with the hexagonal architecture. In any case your user facing access will require identity.

It becomes blurry when your user facing services then call out to other services (via http or message/event), what identity are they using to communicate: user, service or both? You might start looking at RBAC. In any case, this needs to be planned carefully

(4) Persistence / database tier

The prevalent examples you find online , especially in the realm of DBaaS, use just one service identity to access your persistence (file system, database , imp, etc.). So database logs won't tell you who accessed data (RW/RO) and you won't have a chance to implement row level security.

Interestingly this isn't a limitation of databases, they all come with user management, but rather the headache maintaining it or setup another auth

When we designed the Domino REST API, we decided to stick to the JWT based user identity all the way to the database. Apache CouchDB also allows for JWT based authentication. It was lacking the ability to point to an IdP's jwks, so I contributed the CouchDB IdP updater, go check it out.

How do you use identity?

A question that pops up frequently in developer discussions is "how to structure your data in a NoSQL way?". To shed a light on this, we have a look at the approach invented 50 years ago and still an all time favorite

Normalization

In a simple order example, we are looking at four tables:

• Customer
• Product
• Order
• OrderEntry

In this design, there are no duplicates and some simple SQL can list out all I need, for example the order value for a given order:

SELECT oi.order_id,
       SUM(oi.quantity * p.price) AS order_total
FROM OrderItem oi
JOIN Product p ON oi.product_id = p.id
GROUP BY oi.order_id
WHERE oi.order_id = 67111;

or the revenue per customer:

SELECT c.id AS customer_id,
       c.name AS customer_name,
       SUM(oi.quantity * p.price) AS revenue
FROM Customer c
JOIN Order o ON c.id = o.customer_id
JOIN OrderItem oi ON o.id = oi.order_id
JOIN Product p ON oi.product_id = p.id
GROUP BY c.id, c.name;

Development Containers are supposed to liberate your development environment from a specific local installation, like container technology liberated your runtimes (a.k.a YAMLed them into Docker or Kubernetes).

Development != Runtime

Containerization for development has some overlap and quite some difference to containerization for production:

Insights

• There are many getting started resources available: here, here, here, here and here. They are working examples, commonly strong on what and how, but light on why
• There are plenty of templates to learn by example
• There seem to be substantial differences how it works on different platforms, subtle and annoying
• On macOS (14.4.1) with with the devcontainer plugin 0.364.0 mount binds would not work in auxiliary containers, only in the main app
• I couldn't find any descrption which subset of docker-compose.yml is supported
• The most interesting templates, for now, live on the Microsoft Artifact Registry, when you know how to look. Starting with those saves you tons of time and frustration
• You will love code --list-extensions to figure out the extensions you have in your vscode (unless you are a code n00b and don't have any)