wissel.net

Usability - Productivity - Business - The web - Singapore & Twins

vert.x and CORS


One of the security mechanism for AJAX calls is CORS (Cross-Origin Resource sharing), where a server advice a browser if it can request resources from it, coming from a different domain.

It is then up to the browser to heed that advice. To complicate matters: when the browser wants to POST data (or other similar operations), it will go through a preflight request adding to site latency.

I have to admit, I never fully understood the rationale, since only browsers adhere to CORS, any webserver, Postman or CURL ignore CORS happily.

None, One or All, but not Some

There's another trouble with CORS: The specification only allows for no-access, all-access (using * as value for Access-Control-Allow-Origin, with restrictions) or one specific domain, but not a list of domains.

Mozilla writes

Limiting the possible Access-Control-Allow-Origin values to a set of allowed origins requires code on the server side to check the value of the Origin request header, compare that to a list of allowed origins, and then if the Origin value is in the list, to set the Access-Control-Allow-Origin value to the same value as the Origin value.


Read more

Posted by on 07 April 2020 | Comments (0) | categories: Salesforce Singapore

My Maven starter template


Maven is to Java what npm is to JavaScript. It can be a harsh mistress or your best companion. It depends

Beyond dependencies and builds

Maven removes the need to download and manages your dependencies. Unfortunately it doesn't come with mvn install <packagename> like npm (or I haven't learned that yet), so keeping that pom.xml current is a little PITA. However once we make peace with it, the power of plugins makes development in auto-pilot a breeze. Some of the things you can do:

  • Generate a project site
  • Generate various reports: code quality, code coverage
  • Run unit tests

Check out the complete list to get an idea. I'm specifically fond of the site generation capability. It allows us to keep your documentation in the same repository as the project, so we have one place less to worry about.

We simply add /src/site/ to our project and content can be created in multiple formats. My favorite one is Markdown. Besides my handcrafted pages, I generate reports:

  • Issue management
  • Licenses
  • Plugins
  • Source code location
  • Team
  • JavaDoc
  • PMD and CPD
  • Surefire (Test results) and JaCoCo (Test coverage)

All this involved a bit of boilerplate in the pom.xml so I keep a template around,


Read more

Posted by on 06 April 2020 | Comments (0) | categories: Java WebDevelopment XML

eMail etiquette - the 60ties are calling


With WFH being en vogue these days, not only video conferencing and chat, but also eMail.
Dating back to the 1960ties, we had 6 decades to develop etiquette, which seems tobe lost to current users, so here we go again

Addressing

eMail has To, CopyTo (also called CC for Carbon Copy) and BCC (Blind Carbon Copy) as a means of addressing people. They serve distinct purposes:

  • TO: This is the person (or people) we want to act on our message, do something, reply etc. A good email has only few names, ideally one. If we have an ongoing eMail thread that involves multiple actors, we most likely use the wrong channel and are better of using collaborative software like HCL Connections, HCL Sametime, Slack, Teams or Chatter
  • CC: People, we think, who should be keept in the loop. We don't expect any action or reaction of them. A lot of eMail veterans automatically route those messages to a low priority place
    ? BCC: all receipients here get the message and the rest of the addressies won't know. I used to call it the "mobbing copy". BCC is especially fun when someone there hits "reply all" and reveals the readership. There are few legitimate uses for this. One is distribution lists (see below), the other archive/record keeping. Our external readers don't need to know that your compliance archive has the eMail address compliance@acme.com If we really want someone outside the visible thread to take note - forward the message

Subject line

It is like a tweet about our content. The subject needs to justify why it is worth the time and attention to open it. So "Status", "Report" or "Important" don't cut it. Common practise we can see are qualifiers, e.g Opportunity codes or project IDs at the beginning. Something like [T3453] - makes it easier to filter.

The biggest competitor to inbox attention by subject is the sender identity. We probably open a message one or two reporting managers up even with bad subject lines.

Content

Let's keep it crisp and short, best below 5 sentences.

We state:

  • the information we want to provide
  • the exact ask what action we expect, from whom and when
  • name the person "team please look into? " doesn't cut it and is an indicator of a broken process

If there is a lot of information, it might better live in a Wiki, a project place or even a file share. We then provide the news cast overview and a link - Would you like to know more?

There are some interesting cultural differences. In Anglosaxon or Eastern culture we would politely address the person and add a whiff of smalltalk, something along the lines "hope that finds you well". Germans, Dutch and other Nordics consider this a waste of space and time and consider it as the ultimate courtesy to cut through the chase and get to the point.

When we address close co-workers, who value efficiency, it even is OK to skip the greeting. We need to dread carefully here, it needs to be clarified otherwise it is seen ultra rude.

Replies

Do we reply to the sender or all of the addressies together. It seems to be the default for many "replyToAll". This is especially hillarious when a distribution list sneaked on the addressies. The rationale here is: the sender wanted to keep all these in the loop, so I won't break it. For a small group, I hit replyAll, for larger ones only reply.

I would wish the eMail software would warn when you blast a reply. The guardian agrees: don't replyAll.

A special mention: cherry-picking replies. We hit reply all and remove the mailing lists - good. We just remove the project manager we compete with - bad. So we need to be careful of the ramifications. Other receipients might wonder: why are Jane and Joe no longer in this conversation?

Distribution lists

They firmly belong in BCC - avoids ReplyAll armageddon. When we use private distribution list, we need to make sure, they resolve before sending otherwise people can't reply. However - most likely - that group of people would be better served with a shared channel. A good strategy: we put it in BCC, write a two sencent summary and provide a link to the full info. Co-workers who are not into eMail will find in in their [insert the collaborative tool you use]

As usual YMMV


Posted by on 27 March 2020 | Comments (1) | categories: GTD Intercultural

Running Java applications with Notes on macOS


My main work computer is a MacBook running macOS. Thank to Logitech, they keyboard is just fine. As you know macOS neither features Domino Designer or Domino Admin. For recent development I wanted to make sure that my applications can run on the client (I've done that before).

Java, Java on the wall, what's the right path of them all?

(Sorry Schneewitchen)

In Notes/Domino R9.0.1FP8 - (client FP10) the Java runtime was updated to Java8 and in R11 changed to AdoptOpenJDK.

On macOS that led to a particular situation. The jvm packaged with the HCL Notes.app can be found in the path
HCL Notes.app/jre/Contents/Home with bin, lib and lib/ext as we can expect from a JVM. Suspiciously absent are Notes.jar, websvc.jar and njempcl.jar. They can be located at HCL Notes.app/Contents/MacOS.jvm/lib/ext.

While this isn't an issue for the Notes client, it is an obstacle when you try to run an external jar file. java -jar somejar.jar ignores any classpath setting outside the JVM and only loads resources from the default JBM path (lib/ext).

I suspect the separation was neccesary due to the AdoptOpenJDK distribution rules.

To solve this, we can use a start script that creates a symbolic link in the right place. Takeing the usual suspects like DYLD_LIBRARY_PATH and LD_LIBRARY_PATH into account we end with a script like this:

#!/bin/bash
# MacOS Keep Starter file
# Keep locations - update as needed - leave the TLS stuff empty if you don't have it
export KEEPJAR=$HOME/keep/projectkeep.jar
export LOG_DIR=$HOME/keep/logs
export TLSFile=$HOME/keep/private/demoserver.projectkeep.io.pfx
export TLSPassword=supersecret

# Don't change anything below unless you are sure what you are doing
# Java files places unfortunately troublesome, so we link some
cd /Applications/HCL\ Notes.app/jre/Contents/Home/lib/ext
export SRCDIR="../../../../../Contents/MacOS/jvm/lib/ext"
if [ ! -f njempcl.jar ]; then
	ln -s $SRCDIR/njempcl.jar .
    echo "Linked njempcl.jar"
fi
if [ ! -f Notes.jar ]; then
	ln -s $SRCDIR/Notes.jar .
    echo "Linked Notes.jar"
fi
if [ ! -f websvc.jar ]; then
	ln -s $SRCDIR/websvc.jar .
    echo "Linked websvc.jar"
fi

# Local Keep Server
export DEBUG=true
export PATH=/Applications/HCL\ Notes.app/Contents/MacOS:$PATH
export JAVA_HOME=/Applications/HCL\ Notes.app/jre/Contents/Home
export GodMode=true
export DYLD_LIBRARY_PATH=/Applications/HCL\ Notes.app/Contents/MacOS
export LD_LIBRARY_PATH=/Applications/HCL\ Notes.app/Contents/MacOS
echo $LD_LIBRARY_PATH ..
cd $HOME/Library/Application\ Support/HCL\ Notes\ Data
/Applications/HCL\ Notes.app/jre/Contents/Home/bin/java -jar $KEEPJAR
cd ~
echo Done!

This script presumes thay we have admin permissions. We could contemplate to check for existing symbolic links and remove the ones we did set. I decided, that smells too much like YAGNI.

As usual: YMMV


Posted by on 17 March 2020 | Comments (0) | categories: HCL Notes Java macOS

OData - the fine print


For an upcoming event I had to dig a little deeper into OData and learned a few things along the lines.

Same but different

OData's current version is 4.0 and that's the only version I will focus on in this article.

OData is the love child of an effort that joined Microsoft and SAP at the hip to create an universal internet data access standard, that on first sight has a lot of NIH issues to sort out. But I'm sure on a second (or third or forth) look it will all make sense.

Key components of OData

  • a service document in JSON format. It describes what data endpoints are available and where. Depending on the requested scope (more on that later) that JSON contains different levels of details
  • a meta data document. That document is referenced in the service document. The format is XML. It describes the data format found in each record, relations to other record and actions that can be taken. Looks like XML Schema, but not quite. Record is used loosely here, data can be hierarchical and nested
  • a query language squeezed into an URL, so data can be fetched using HTTP GET without using a body submitted. Query allows data selections, filtering, output size and sorting to be specified
  • Query results in JSON format enhanced with OData specific Metadata

What I liked a lot: OData.org offers a rich set of documentation and tutorials, including a Postman collection to learn

Happy soup

So dealing with OData you can enjoy a mix of technologies:

  • JSON for service document and payloads
  • XML for meta data
  • a dot notation (like dotNet or Java) for property values like data types (go learn about it). E.g. Edm.String defines a String value or Org.OData.Core.V1.ResourcePath the path to data
  • another query language
  • individual items by default get addressed in an uncommon format: serviceurl/People('johndoe'). Luckily this is optional and you can use the more common format for a route: serviceurl/People/johndoe. This seems to be born out of squezing the square pin URL into the round hole of database schemas and flexible primary keys

On the upside: Your Salesforce and Excel users will love it. Import and CRUD operations are well supported.

Initially I was looking at Apache Olingo to implement my needs. Unfortunately the library is too close to the Java servlet model and static defined data models (like your one application, not your data platform), so I disected my needs the hard way.

Lessons learned along the way

  • A OData client (think Excel, Tableau, Salesforce) will initially connect to the service document url. It sends a GET request. To specify what level of detail is required the ACCEPT header is used. So when testing your provider implementation you need to take that into account
  • Excel sends ACCEPT application/json;odata.metadata=minimal
  • Salesforce sends ACCEPT application/json;odata.metadata=full
  • OpenAPI can specify the top level application/json or the full qualified content type in the response. If you fully qualify it, you might end up with a 406 and wonder what happens
  • The meta data url gets specified in your service document and is usually the service document url extended by /$metadata
  • I ran into a potential Salesforce bug, where the meta data URL was ignored and the service document URL extended by $metadata. Note the missing /. Caused a bit of grief since Excel does /$metadata
  • Everything is case sensitive, which makes sense, but is fun in Domino that treats item names case insensitive
  • I couldn't get Salesforce to play nice until I added a field ExternalId to my form. Might be my lack of understanding how to change the primary key Salesforce will play nice with or something missing in my Metadata

Same, same, but different

Swap out the GET for a POST and the XML for something not-quite-JSON and you get GraphQL that tries to address a similar set of functionalities. We really love wheels reinvented

More lessons to be learned, stay tuned!


Posted by on 04 March 2020 | Comments (0) | categories: OpenSource WebDevelopment

Domino Administration Back to Basics (Part 2) - Networking


In Part 1 we learned about the marvelous world of Notes Names, X400 and the perils of messing with certificates. One big difference to X509 is the (almost) absence of certificate Command Line tools that can be so much fun.

Domino Networking - protocols as you like it

Domino predates the rise of TCP/IP and the internet. To no surprise it has its own idea about networking. Starting with protocol support:

  • Netbios using NDIS (doesn't route) and
  • IPX/SPX A protocol from days long past, when red boxes weren't Redhat but Novell
  • X.PC DialUp - Yes. A modem or something that takes modem commands and will establish a serial connection, no longer ships with Notes
  • A few more obscure protocols: Vines, SPXII
  • last not least TCP/IP

Having this zoo of protocols, Notes needs its own version of name resolution. That version is called Notes Named Network

One step back: What makes a Notes Domain?

A Notes Domain consists out of one or more server that use a Domino directory (a.k.a Public Name & Addressbook a.k.a names.nsf) with the same replica id (a story for another time) as the other member servers and have the same Domain name in their server document (that's where most of the server's setting are stored).

A popular point of confusion: Notes Names (from Part1) and Notes Domains: It is quite common to name your Domain after your orgID, but not mandatory. SO you could have HeavyRock/Acme@Acme or Sandstone/Acme@ToonsInc or Machine/Blowup@Acme The first and the last would be in the same Domain, while the first and second share the Org certifier. Anything goes, but to keep it simple, keep OrgId and Domain the same - unless you have 5 good reasons not to.

Another one: NEVER name your Notes Domain so it could be mistaken for an internet Domain. So no . in the name. Spaces interestingly are OK!


Read more

Posted by on 05 February 2020 | Comments (2) | categories: Domino Networking

Domino Administration - Back to Basics (Part 1) - Certificates


Domino is different, a lot of its concepts predate the internet and quite often inspired the standards. This is not an step-by-step instruction, but an introduction into concepts. The "step by step" approach is another story for another time.

In the beginning was the Certificate

Notes and Domino run using ID files. This are not merly files that can arbitrarily reconstructed, but cryptographically created Public Private key pairs. To avoid naming collisions the key names are hierarchical (Since R2), so anyone can call their server Server1 without confusion (sort of). This hierarchy is achieved using X400 naming conventions, an early competitor to DNS naming. A X400 name can consist out of multiple parts, these are the ones Domino is using:

X500 Name

So the minimum is a common name and an Org. The starting point of each Domino journey is the creation of the OrgID. All other parts depend on it. Note: there isn't a country ID, even if country mentioned after the Org. When you create your OrgID (while setting up the very first server), you can specify the country and it becomes part of the OrgID.

In practise however I haven't seen many OrgIDs that would carry a country code in them. So you can skip that part.

Signing Ids

Using the OrgID as signing certificat, one can go and create server and user certificates. E.g using the OrgID /O=Acme one can create /CN=Coyotee/O=Acme. For convenience the qualifiers are usually omitted, their meaning results from the position (and the fact that country is 2 letter if present and Orgs have 3 or more). So instead of /CN=Coyotee/O=Acme one can write Coyotee/Acme.

In practise however more enlightened organisation use their OrgCert as cautiously as root certificates in the X509 world and only register/sign Organizational Unit (/OU=) Ids which then can be used to sign server and user certificates.

Trust between certificate is hierarchical, similar to Internet certificates, so IDs having the same root (/O) certifier recognizse each other. The hierachy can be used in Access Control (another story for another time) to grant access to all IDs at a given level e.g. /OU=Management/O=Acme


Read more

Posted by on 04 February 2020 | Comments (3) | categories: Domino

Generating JWT tokens for tests


There are many options for Authentication and Authorisation. I'm fond of Json Web Tokens (JWT), implementing RFC7519. Mostly because they are like LTPA for grownups, but standard compliant

Quis custodiet ipsos custodes?

JWT contains information that is digitally signed (and optional encrypted), so a receiving end can verify that the information is correct. The key elements here are:

  • the JWT contains a claim, at least the subject that is tamper resistant by being digitally signed
  • JWT issuer and JWT consumer trust each other, by either having a shared secret (bad idea) or using a public/private key pair. The issuer signs the information with a private key. The consumer (your application server) verifies the signature using the public key

So besides protecting the private key of the issuer, you also want to be clear who's keys you trust. The one who holds your identity can impersonate you at any time (so you might rethink if "Login with Facebook" is such a brilliant idea after all).

However when testing systems, you develop local, you want to be able to have any user at your disposal, so you can generate the claim that is the Open Sesame to your test regime

Building a claim

We start by building the raw claim template.json file:

{
  "iss": "Alibaba Caves",
  "aud": "40Thiefs",
  "expSeconds": 300
}

It contains an issuer, an audience and the duration in seconds. The later one is for my convenience. The receiving system might or might not check issuer (iss) and/or audience (aud).

Next step is to create a public/private key pair


Read more

Posted by on 01 February 2020 | Comments (0) | categories: JavaScript WebDevelopment

Running a CalDAV server on Ubuntu (2020 edition)


When playing with calDAV it makes sense to have a reference implementation to refer to. The GOLD standard ist Apple's CalendarServer with its lovely Open Source Repository.

Getting it to work on Linux (Ubuntu in my case), isn't for the faint of heart. Here is what you need to do.

File system preparation

Calendar server needs extended attributes. The default Ext4 file system does support them out of the box, but you want to check first (you might have a different file system after all):

Check if file system supports extended attributes

Check using this command:

touch test.txt
setfattr -n user.test -v "hello" test.txt
getfattr test.txt

Expected output: user.test

If the command setfattr is not available, install with sudo install attr

Enable extended atttributes if required

Important if the previous command worked, skip this step. Don't mess with fstab! Make a backup copy if required and have your emergency boot stick ready. A messed up fstab will prevent your machine from booting!

Edit /etc/fstab. In the 4th colum of the file system add user_xattr. There might be values like noatime or defaults. Add user_xattr separated by a comma. Reboot.

Install calendar server

Execute sudo apt install calendarserver postgresql.

The server wants a postgresql database, so you need to be sure to have that installed too. Apple loves Python, so there ill be quite some phython packages installed, as well as a new user caldavd Since this is a system user, its home directory is /var/spool/caldav


Read more

Posted by on 01 February 2020 | Comments (0) | categories: calDAV WebDevelopment

Unit Tests and Singletons


Good developers test their code. There are plenty of frameworks and tools around to make it less painful. Not every code is testable, so you write test first, to avoid getting stuck with untestable code. However there are situations where Unit Testing and pattern get into each other's way

The singleton and test isolation

I'm a big fan of design patterns, after all they are a well-proven solutions for specific known problems. E.g. the observer pattern is the foundation of reactive programming.

A common approach to implementing cache is the singleton pattern, that ensures all your code talks to the same cache instance ,independent of what cache you actually use: Aerospike, Redis, Guava, JCS or others.

Your singleton would look like this:

public enum TicketCache {
  INSTANCE;
  
  public Set<String> getTickets(final String systemId) {
    Set<String> result = new HashSet<>();
    // Your cache related code goes here
    return result;
  }
  
  public TicketCache addTicket(final String systemId, final String ticketId) {
    // Your cache/persistence code goes here
    return this;
  }
}

and a method in a class returning tickets (e.g. in a user object) for a user could look like this:

  public Set<String> getUserTickets() {
    Set<String> result = new HashSet<>();
    Set<String> systemsResponsibleFor = this.getSystems();
    systemsResponsibleFor.forEach(systemId -> 
      result.addAll(TicketCache.INSTANCE.getTickets(systemId)));
    return result;
  }

Now when you want to test this method, you have a dependency on TicketCache and can't test the getUserTickets() method in isolation. You are at the mercy of your cache implementation. But there is a better way


Read more

Posted by on 10 January 2020 | Comments (1) | categories: Java UnitTesting