File handling ​

Compas also comes with various utilities across the stack to handle files in a consistent way.

Generated router & validators ​

Let's start with looking at the code generators. Here we have the T.file() type to represent files as can be seen in the following examples:

const T = new TypeCreator();
const R = T.router("/");

R.post("/upload")
  .files({
    myFile: T.file(),
  })
  .response({ success: true });

R.get("/download").response(T.file());

Files are handled separately by the generator and validators, and are put on ctx.validatedFiles with help from formidable. In the generated api clients we generate the correct type (ReadableStream or Blob) depending on the context. And allow for setting custom file parsing options createBodyParsers provided by @compas/server

Saving files ​

@compas/store comes with Postgres and S3 which we combine in the various utilities for working with files.

fileCreateOrUpdate ​

Creates a new file and stores it in both Postgres and S3. If an existing id is provided the file is overwritten. This function only requires a file name and the source and is able to infer contentType and contentLength. If allowedContentTypes is provided, an error will be thrown if the inferred content type is not one of the allowed content types.

Example

/**
 *
 * @param {InsightEvent} event
 * @param {AppSaveFileFiles} files
 * @return {Promise<void>}
 */
export async function appSaveFile(event, files) {
  eventStart(event, "app.saveFile");

  await fileCreateOrUpdate(
    sql,
    s3Client,
    {
      bucketName: "my-bucket",
      allowedContentTypes: ["image/png", "application/x-sql"],
      schedulePlaceholderImageJob: true,
    },
    { name: files.uploadedFile.originalFilename },
    files.uploadedFile.filepath,
  );

  eventStop(event);
}

For placeholder image generation, the jobFileGeneratePlaceholderImage needs to be used.

Errors:

• file.createOrUpdate.invalidName -> When name is not specified.
• file.createOrUpdate.invalidContentType -> When the content type is not one of allowedContentTypes.

Securing file downloads ​

In some cases you want to have private files as well, you can accomplish this by using fileSignAccessToken and fileVerifyAccessToken. When returning an image url to the client, you can add a JWT based token to the url specific for that file id, and with a short expiration date via fileSignAccessToken. Then, when the user requests the file, fileVerifyAccessToken can be used to check if the token is still valid and issued for that file id.

Let's look at a quick example;

Definition:

const T = new TypeCreator();
const R = T.router("/");

R.get("/product", "getProduct").response({
  publicImageUrl: T.string(),
  privateAvatarUrl: T.string(),
});

R.get("/product/public-image", "publicImage").response(T.file());

R.get("/product/private-avatar", "privateAvatar")
  .query({
    accessToken: T.string(),
  })
  .response(T.file());

Implementation:

// For the example :)
const [publicImage] = (
  await queryFile({
    where: {
      id: uuid(),
    },
  }).exec(sql)
)[0];
const [privateImage] = (
  await queryFile({
    where: {
      id: uuid(),
    },
  }).exec(sql)
)[0];

appController.getProduct = (ctx) => {
  // Do user checks here, so see if the privateAvatarUrl should be added.

  ctx.body = {
    publicImage: fileFormatMetadata(publicImage, {
      url: "https://example.com/product/public-image",
    }),
    privateImage: fileFormatMetadata(privateImage, {
      url: "https://example.com/product/private-image",
      signAccessToken: {
        signingKey: "secure key loaded from secure place",
        maxAgeInSeconds: 2 * 60, // User should load the image in 2 minutes
      },
    }),
  };
};

appController.publicImage = async (ctx) => {
  const file = await queryFile({ where: { id: publicImage.id } }).exec(sql);

  await fileSendResponse(s3Client, ctx, file);
};

appController.privateAvatar = async (ctx) => {
  const file = await queryFile({ where: { id: privateAvatar.id } }).exec(sql);

  // Throws if expired or invalid
  fileVerifyAccessToken({
    signingKey: "secure key loaded from secure place",
    expectedFileId: file.id,
    fileAccessToken: ctx.validatedQuery.accessToken,
  });

  await fileSendTransformedImageResponse(sql, s3Client, ctx, file);
};

An important note is that the tokens can't be revoked. So if you have that requirement there are two options;

• Keep a blacklist of tokens somewhere
• Regenerate the signingKey, rendering all tokens invalid.

Unified file responses ​

For a more complete metadata object to return from your api's, you can use StoreFileResponse. It contains various properties, like the content type, name, and if applicable, the image placeholder. To format this response object, you could call fileFormatMetadata. It allows for formatting a secure file url as well via its accepted options.