Release notes v0.0.115 #

This release is not that big, but still worth for proper release notes. We go over the changes package by package.

Code gen #

It does not feel like a proper Compas release without some fixes or features added to the code generators, as is the case this time. The table traverser is a relative new feature added in Compas v0.0.103. At that point we did not have a query builder, and generated a stand-alone table traverser that unlocked the following chains:

// Get all users that liked the specific post
const users = queryUser().viaLikes().viaPosts({ id: postId });

This was straight forward to use, but limited in options. You could not use this to filter on two separate ' traverse'-chains.

Next came the query-builder with its built-in support for table traversal. This unlocked traversing via multiple chains, however it was not documented as such. To explain why, let us do a run down of what the traverser does.

Provided the following structure:

User:
- id: uuid
- name: string

Post:
- id: uuid
- writer: uuid -> User
- title: string

Like:
- id: uuid
- user: uuid -> User
- post: uuid -> Post

We have User, Post and Like as our tables. The relation being that a User is the writer of many Post, and a User can Like many Post. To get all users that have written at least a single post, we can do the following:

const users = await queryUser({
  where: {},
  viaPosts: {},
}).exec(sql);

Internally, the viaPosts was converted to a QueryPart and thus allowed to be set on the where-object as the idIn field. The QueryPart was a query selecting the writer field from Post.

If we do one of the following things you would get unexpected results:

// Get users that have written a post and that are in our selection in 'idIn'.
const users = await queryUser({
  where: {
    idIn: ["uuid-110", "uuid-120", "uuid-130"],
  },
  viaPosts: {},
}).exec(sql);

// Get users that have written a post and have liked an post
const users = await queryUser({
  where: {},
  viaPosts: {},
  viaLikes: {},
}).exec(sql);

We have a problem in both cases. In our first case we already set idIn however viaPosts will overwrite it. In the second case, since both Like and Post are referenced via the id on a User, viaLikes would overwrite viaPosts since they both set directly the idIn property on the provided where object.

Of course not all references would go via the idIn parameter, so indirectly the generated traverser could work with multiple chains, but would fail on the most useful cases.

In this release we solve these cases. We now generate an INTERSECT query from all viaXxx calls and if necessary combine it with an existing idIn array. INTERSECT gives us the same AND guarantees as normal where filters.

Store #

The JobQueueWorker got two upgrades as well, breaking existing behaviour.

Managed retries:

The worker now won't leak any errors that are thrown by job handler invocations. Instead, all errors are caught. On error the job id, name, retryCount and error are logged via an error log. The retryCount is then increased and sql transaction is committed.

By adding a savepoint, we can rollback any queries done by the job handler. Note that you won't see failed transactions from the worker anymore, so that metric may become less relevant for you. We added a maxRetryCount to the options that can be provided to the constructor, it defaults to 5.

Handler timeouts

To prevent that the worker is too busy with a job, comes in a infinite async loop or for some other reason takes long to complete. We now automatically stop the job and let it be processed as if an error was thrown by the handler. Which results in an increased retryCount. Note that this won't work correctly if the job does only synchronous processing.

The timeout can be configured via the constructor options under handlerTimeout in milliseconds, and defaults to 30 seconds.

In closing #

Some other chores are being done as well, like writing docs (A). The Compas repo will now require use of conventional-commit like specifiers in the commit title, like feat, fix, etc. This was decided after some feedback to get a quicker overview of the current state of development.

I am also experimenting a bit with JSDoc based documentation again. We had TSDoc based documentation for a while, based on the index.d.ts files that we provide. The output was too unstructured, and overall hard to use in any form. If anyone has any suggestions please let me know directly or via the Align JSDoc issue.