Release notes v0.0.83

As I already apologized in advance in the previous release notes, let's get started! Since we have some breaking changes to the structure format you should update in the following order:

  • Packages
  • Backends
  • Generated clients

Note that some parts of this document can be a bit hard to read through, but take it as a reference to explore the newly generated features.

Code gen

The code generators got a complete refactor. We did this to allow some planned features, make types more correct, and all in all just give it a nice cleanup. This allowed us to make the validators more tree shakeable and have a more extensive sql query setup in the future. We also dropped support for generator plugins and type plugins. Instead, we are going to aim for structure stability so that anyone can generate whatever they want, in whatever language they want.

This all probably breaks your code, so below a list of situations and how to migrate them over.

Generated clients

It is not necessary anymore to pass in collectValidatorErrors to app.generate() calls. We now always collect all errors. In the near future we are planning to collect an error set, so you are able to statically check if you handle all possible cases.

Before generating again, make sure to remove the current generator output. We are now able to generate TypeScript files for all the generators, which is enabled by default.

A lot of generated types are not available anymore. However, the types that are generated should be more relevant, for example dates are now be handled correctly.

When you want to use validators, make sure to import them from the specific file, ( eg ./generated/my-group/validators.ts). This will hopefully allow Webpack to strip out any unused validators.

The generated api client and react-query based hooks should still work the same.

Packages

Packages that use queries should allow overriding the generated queries. This can be archived with something like the following in src/generated.js:

// An example taken from @lbu/store
import { queries as defaultQueries } from "./generated/index.js";

export let queries = defaultQueries;

/**
 * Overwrite used generated queries.
 * This is needed when you want cascading soft deletes to any of the exposed types
 * @param {typeof defaultQueries} q
 */
export function setStoreQueries(q) {
  queries = q;
}

This way the package can still do cascading soft deletes across user code if necessary. The package should then use import { queries } from "./generated.js"; instead of import { xxxQueries } from "./generated/queries.js";

Backends

The generated router & validators work practically the same as the previous generated versions. The most notably change here is that the exports go via ./generated/index.js.

The big hit here is with queries and structure changes. T.objects that have enabled queries are changed the most:

We automatically add primary keys of type 'uuid' if none is provided. This can be disabled by passing in withPrimaryKey: false in to .enableQueries().

Another new option is added to .enableQueries(), 'isView'. This disables some parts of the generator that have to do with INSERT, UPDATE and DELETE queries.

The sql relation system is revamped. They now need to be provided via T.object().relations(), which accepts any number of relations. To create relations the TypeCreator (T.) supports the following:

  • T.oneToMany("key1", T.reference()): The '1' side of a 1 - many relation
  • T.manyToOne("key2", T.reference(), "key1"): The 'many' side of many - 1 relation. This side will get an extra field that has a foreign key to the primary key of T.reference().
  • T.oneToOne("key3", T.reference(), "key4"): The owning side of a 1 - 1 relation. The object where this relation is added will get an extra field that has a foreign key the primary key of the provided T.reference(). The inverse side is automatically added.

Both T.manyToOne() and T.oneToOne() support .optional(). This results in a nullable field, and the generated structure will by default add an ON DELETE SET NULL. The error messages have some improvements here as well, so if something is missing, the generator will let you know.

The output of the 'sql' generator does not contain everything from the previous generator. The Upsert queries and selects with joins (eg. userSelectWithPosts) are missing. However, most of the time the generated queries should be compatible with the new output, so it is fine to keep the previous generated/queries.js for a bit while migrating those queries to the business domain.

The sql output is now modular build. This is allowed by a custom query builder provided by @lbu/store. More on that builder later in these release notes. The generated queries also do not use the 'group' name in any of the output. This results in more exported functions for you to use. Say that we have an object called 'user', we get the following functions:

Partials:

  • userFields(): QueryPart: Returns qualified names for all field of the user table. This can be used in SELECT queries and for RETURNING part in INSERT/UPDATE queries.
  • userWhere(): QueryPart: Builds a WHERE statement dynamically. This has some new options like isNull, isNotNull , notEqual, notLike and notIn.
  • userOrderBy(): QueryPart: An appropriate default ORDER BY statement for the specific table. Also supports tables without any of withDates and withSoftDeletes enabled.
  • userInsertValues(): QueryPart: Creates the VALUES part of an INSERT query, making sure that the defaults provided by .default() are all taken into account.
  • userUpdateSet(): QueryPart: Builds the SET part of an UPDATE query. Making sure updatedAt is added when necessary.

Basic CRUD queries:

  • userSelect(sql, where): Execute a SELECT query
  • userCount(sql, where): Count the affected rows and returns a number
  • userInsert(sql, where): Execute an INSERT query, will return the affected rows.
  • userUpdate(sql, where): Execute an UPDATE query, will return the affected rows.
  • userDelete(sql, where): Execute a DELETE query. When the option withSoftDeletes is true, this will be an UPDATE query.
  • userDeletePermanent(sql, where): Only generated when the option withSoftDeletes is true, this will execute a DELETE query and is thus destructive.

The 'delete' queries in a withSoftDeletes situation will now cascade this automatically to the dependent relations. It is advised to always run this in a transaction.

Traverse the graph:

The last 'query' added is traverseUser(). This is a set of functions that are able to get relations via other relations. Say we have Post <-> User <-> Group, we can easily fetch all posts created by users in the group of a specific user, with only a single 'id' of the user, like so: traverseUser({ id: "my id" }).getGroup().getUsers().getPosts(). Along the way you can do some more filtering. Future versions may add features like limiting the amount of results per step.

We also revamped the dumpStructure option that could be passed to app.generate(). It will now create a file as well in the output directory. This file should not be committed to version control, and is still only an example to help you with migrations.

As noted in the 'Packages' part of this changelog, it is advisable to call any setPackageQueries of the packages you are using.

The last lovely feature that is added, is stable fields output. This makes sure the date fields added by withDates and withSoftDeletes are always last, and the (injected) primary key is always first.

Store

The store got some features added. We migrated to the new generated queries without any breaking changes.

We now export a query function this is a tagged template function, and thus supports to be called like the following:

const bucketName = "bar";

// Typeof myQuery = `QueryPart`, this is the same as returned by the 'partial' generated queries
const myQuery = query`SELECT ${fileFields()} FROM "file" f WHERE f."bucketName" = ${bucketName}`;
myQuery.append(query`ORDER BY ${fileOrderBy()}`);

const queryResult = await myQuery.exec(sql);

The query function is used by all newly generated queries, and allows injection safe query building. It also supports providing another QueryPart as an injected value as well as .append-ing one.

Another neat function added is explainAnalyzeQuery. This can run any query safely (in a rolled back transaction) and return the query plan as either text or json. Note that the result you get on a development setup is most likely not the same as the one you will get in production. So keep that in my mind while exploring 😃

Lastly there is now support for grouping and ordering files. The two entities added are fileGroup and fileGroupView. As the name suggests fileGroupView is backed by a view, and only adds the calculated isDirectory property. Most operations should be done via the generated queries, however there are three functions exported that will help you with your operations:

  • hoistChildrenToParent: move all children a level up in the tree. Most likely used in preparation of deleting a group.
  • updateFileGroupOrder: given an array of id's atomically updates the order to reflect the order of the array, without the need of a transaction. Note that the provided id's do not necessarily be of the same parent, and thus any number of groups can be sorted in a single call.
  • getNestedFileGroups: returns the nested structure of a group starting at any arbitrary root, and able to include the referenced files. The resulting nested structure is also sorted.

There is also support for an arbitrary meta field on any file and fileGroup like so:

const app = await App.new();
const T = new TypeCreator("store");

app.add(
  T.object("fileMeta").keys({
    // Put your custom meta data here
  }),
);

app.add(
  T.anyOf("fileGroupMeta").values(
    {
      // Custom metadata
    },
    {
      // Or another structure with different meta data
    },
  ),
);

When migrating from existing implementations of grouping, it is advisable to change all relations to point to T.reference("store", "file") and places that need a group to T.reference("store", "fileGroup").

In closing

This release contains probably the most breaking changes till now. We are aiming for structure stability and a first beta on 0.1.0 in the upcoming weeks. Till then, expect some new query capabilities like atomic updates, a query builder, many to many relations, unique index and upsert support and many more.

Dang, my vacation was exactly long enough to complete this, but it's over now -_-.