File handling
Compas also comes with various utilities across the stack to handle files in a consistent way.
Generated router & validators
Let's start with looking at the code generators. Here we have the T.file()
type to represent files as can be seen in the following examples:
const T = new TypeCreator();
const R = T.router("/");
R.post("/upload")
.files({
myFile: T.file(),
})
.response({ success: true });
R.get("/download").response(T.file());
Files are handled separately by the generator and validators, and are put on ctx.validatedFiles
with help from formidable. In the generated api clients we generate the correct type (ReadableStream
or Blob
) depending on the context. And allow for setting custom file parsing options createBodyParsers
provided by @compas/server
Saving files
@compas/store
comes with Postgres and minio which we let work together in the various utilities for files.
createOrUpdateFile
Creates a new file and stores it in both Postgres and Minio (S3). If an existing id
is provided the file is overwritten. This function only requires a file name and the source and is able to infer contentType
and contentLength
. If allowedContentTypes
is provided, an error will be thrown if the inferred content type is not one of the allowed content types.
Example
/**
*
* @param {InsightEvent} event
* @param {AppSaveFileFiles} files
* @return {Promise<void>}
*/
export async function appSaveFile(event, files) {
eventStart(event, "app.saveFile");
await createOrUpdateFile(
sql,
minio,
"myBucket",
{ name: files.uploadedFile.originalFilename },
files.uploadedFile.filepath,
{
allowedContentTypes: ["image/png", "application/x-sql"],
},
);
eventStop(event);
}
Errors:
store.createOrUpdateFile.invalidName
-> When name is not specified.store.createOrUpdateFile.invalidContentType
-> When the content type is not one ofallowedContentTypes
.
Securing file downloads
In some cases you want to have private files as well, you can accomplish this by using fileSignAccessToken
and fileVerifyAccessToken
. When returning an image url to the client, you can add a JWT based token to the url specific for that file id, and with a short expiration date via fileSignAccessToken
. Then, when the user requests the file, fileVerifyAccessToken
can be used to check if the token is still valid and issued for that file id.
Let's look at a quick example;
Definition:
const T = new TypeCreator();
const R = T.router("/");
R.get("/product", "getProduct").response({
publicImageUrl: T.string(),
privateAvatarUrl: T.string(),
});
R.get("/product/public-image", "publicImage")
.params({
id: T.uuid(),
})
.response(T.file());
R.get("/product/private-avatar", "privateAvatar")
.query({
accessToken: T.string(),
})
.response(T.file());
Implementation:
// For the example :)
const publicImageId = uuid();
const privateAvatarId = uuid();
appController.getProduct = (ctx, next) => {
// Do user checks here, so see if the privateAvatarUrl should be added.
ctx.body = {
publicImageUrl: "https://example.com/product/public-image",
privateAvatarUrl: `https://example.com/product/private-avatar?accessToken=${fileSignAccessToken(
{
fileId: privateAvatarId,
signingKey: "secure key loaded from secure place",
maxAgeInSeconds: 2 * 60, // User should load the image in 2 minutes
},
)}`,
};
return next();
};
appController.publicImage = async (ctx, next) => {
const file = await queryFile({ where: { id: publicImageId } }).exec(sql);
await sendFile(ctx, file /* ... */);
return next();
};
appController.privateAvatar = async (ctx, next) => {
const file = await queryFile({ where: { id: privateAvatarId } }).exec(sql);
// Throws if expired or invalid
fileVerifyAccessToken({
signingKey: "secure key loaded from secure place",
expectedFileId: file.id,
fileAccessToken: ctx.validatedQuery.accessToken,
});
await sendFile(ctx, file /* ... */);
return next();
};
An important note is that the tokens can't be revoked. So if you have that requirement there are two options;
- Keep a blacklist of tokens somewhere
- Regenerate the
signingKey
, rendering all tokens invalid.