CommunitySolidServer/test/unit/util/locking/GreedyReadWriteLocker.test.ts
jaxoncreed 1d65143e89
feat: Add identity provider (#455)
* Add identity provider handler as a dependency

* Temp Identity

* Figured out how to get koa to work

* Hooked up idp to networking

* Feat/idp architecture refactor (#430)

* Logs in with solid oidc

* Refactored Provider

* Attempt to hook up dependencies

* Partial wiring of oidc provider components

* IdP networking now works with architecture

* Interaction Handlers Set Up

* fix: Rename & adapt to CSS

* Included Login Interaction

* Refactored architecture to bind Interaction Policy to HttpHandlers

Co-authored-by: Matthieu Bosquet <matthieubosquet@gmail.com>

* fix: Rebase on master

* fix: DI after rebase

* Reimplemented Routing

* Renamed modules and removed ProviderFactory (#450)

* refactor: Solid IdP DI

* refactor: IdP interaction handler DI

* refactor: IdP interaction waterfall

* refactor: Remove unnecessary legacy URL parse

* fix: Add legacy parse back in

* feat: adapter & fix: handlers

* Removed adapter factory

* fix: refactor IdP

* fix: refactor IdP

* fix: refactor IdP

* feat: Add IdP to file storage config

* fix: Unintended commit

* fix: Components ignore

* feat: Basic resource store adapter

* Partially complete idp routing

* Set up initial routing injection graph

* Clean up ResourceStorageAdapter

* Refactored configuration architecture

* Hooked up Login UIs (#518)

* feat: Use template path & run fileserver

* feat: Use util function to read resource

* Fixed DI JSON-LD context

* fixed rendering

* WebId validator

* Set up persistent storage for loing and register

* Fixed ejs template routing

* Refactored StorageAdapters

* NSS login successful

* Forgot password infrastructure

* Can send email (#557)

* Can send email

* fix: IdP crashes if interaction ID doesn't exist (#587)

* feat: Require an issuer registration token

* fix: Issuer registration token typo in error

* fix: Remove dummy IdP storage adapter

* fix: Remove unused library lodash

* fix: Remove unused library lru-cache

* Production ready keystore

* Ruben comments before clownface removal

* Removed clownface

* Change key value store

* Completed Ruben's comments

* Added comments to each class

* Fixed errors on login

* Ruben feedback

* Refactored out getPostRenderHandler

* Identity provider tests (#622)

* corrected tests lacking <void> on promises

* Added files for all idp tests

* Added unfinished tests for all added files

* ErrorHandlingWaterfallHandler

* RenderEjsHandler and RouterHandler tests

* GetPostRouterHandler and BasicOnErrorHandler tests

* Corrected tests for updates to Idp

* fix: missing export

* fix: audience claim

* Client Id Support (#630)

* Added client_id for the auth challenge

* Update src/identity/storage/ClientWebIdFetchingStorageAdapterFactory.ts

Co-authored-by: Matthieu Bosquet <matthieubosquet@gmail.com>

Co-authored-by: Matthieu Bosquet <matthieubosquet@gmail.com>

* fix: Rebase fixes

* Several minor Idp changes/refactors (#656)

* fix: Minor changes

* refactor: Split EmailPasswordInteractionPolicy

* refactor: Remove ErrorHandlingWaterfallHandler

* refactor: Clean up dependencies

* fix: Add dummy IdentityProviderHandler to fix integration tests

* Replace KeyValueStore with KeyValueStorage (#663)

* feat: Create WrappedExpiringStorage

* refactor: Update ResourceStoreEmailPasswordStore to use KeyValueStorage

* refactor: Update KeyGeneratingIdpConfigurationGenerator to use KeyValueStorage

* refactor: Update ResourceStoreStorageAdapterFactory to use ExpiringStorage

* refactor: Removed KeyValueStore

* refactor: Simplify EmailPassword handlers (#664)

* refactor: Order index.ts

* test: Add EmailPasswordForgotPasswordHandler unit tests

* test: Add EmailPasswordGetResetPasswordHandler unit tests

* test: Add EmailPasswordLoginHandler unit tests

* test: Add EmailPasswordRegistrationHandler unit tests

* test: Add EmailPasswordResetPasswordHandler unit tests

* test: Remove unnecessary test file

* feat: Basic instructions for using the IdP

* fix: IdP instructions and add example WebID

* fix: IdP registration copy

* fix: IdP instruction editorial

* Update README.md

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* Update README.md

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* test: Add KeyGeneratingIdpConfigurationGenerator unit tests

* test: Add KeyValueEmailPasswordStore unit tests

* test: Create IdP integration test

* test: Add EmailPasswordInteractionPolicy unit tests

* test: Add BasicIssuerReferenceWebIdOwnershipValidator unit tests

* test: Add ChooseInitialInteractionHandler unit tests

Also fixes the config warning.

* test: Add EjsTemplateRenderer unit tests

* test: Add EmailSender unit tests

* test: Add FormDataUtil unit tests

* test: Add IdpRouteController unit tests

* test: Add OidcInteractionCompleter unit tests

* refactor: Simplify ClientWebIdFetchingStorageAdapterFactory

* test: Add ClientWebIdFetchingStorageAdapterFactory unit tests

* refactor: Fix ejs html warnings

* test: Add step to test logging in again

Included are updates to handle cookies more correctly.

* feat: Add IdpConfirmHttpHandler

This way there's a handler for the confirm page.

* test: Add ExpiringStorageAdapterFactory unit tests

* test: Add IdentityProviderFactory unit tests

* test: Add IdentityProviderHttpHandler unit tests

* refactor: Minor refactors

* refactor: Use jose instead of node-jose

* refactor: Use jose instead of node-jose

Reduces the number of dependencies since other libraries
also depend on jose.

* Update src/identity/configuration/KeyGeneratingIdpConfigurationGenerator.ts

Co-authored-by: Matthieu Bosquet <matthieubosquet@gmail.com>

* refactor: Use interfaces instead of abstract classes

* refactor: Make WebIdOwnershipValidator an AsyncHandler

* refactor: Make TemplateRenderer an AsyncHandler

* fix: Fix typing issue

* fix: Convert JWK to plain object for node 15

* feat: Update CI configuration

--ignore-scripts was removed because it also stopped dependency scripts,
which was a requirement for bcrypt.
15.0 was removed since that version doesn't run the required scripts after install.
14.0 was removed since the somehow it caused the solid-authn client to do the wrong calls.

* test: Run integration tests on Node 14.2

This is the lowest 14.x version where the IdP integration tests succeed.

* feat: Use ErrorResponseWriter for handling oidc errors

* test: Mock Date in OidcInteractionCompleter tests

* fix: Correctly generate new identifiers

Previously there could be double slashes if the base URL ended in slash.

* fix: Correctly handle storagePathName in ExpiringStorageAdapterFactory

* fix: Fix issue with new CliRunner test in rebase

* fix: Handle unknown errors more consistently

* feat: Make idp parameter dynamic

* feat: Add more logging

* refactor: Link css instead of injecting

* fix: Fix redis integration tests with idp

* refactor: Shorten idp class names

* refactor: Remove e-mail configuration from default config

* feat: Store JsonResourceStorage data in a single container

* feat: Make sure expired data gets removed at some point

* feat: Only accept strings as keys in KeyValueStorage

* fix: Various minor fixes based on review

Co-authored-by: Matthieu Bosquet <matthieubosquet@gmail.com>
Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2021-05-04 13:17:43 +02:00

314 lines
11 KiB
TypeScript

import { EventEmitter } from 'events';
import type { ResourceIdentifier } from '../../../../src/ldp/representation/ResourceIdentifier';
import type { KeyValueStorage } from '../../../../src/storage/keyvalue/KeyValueStorage';
import { ForbiddenHttpError } from '../../../../src/util/errors/ForbiddenHttpError';
import { InternalServerError } from '../../../../src/util/errors/InternalServerError';
import { GreedyReadWriteLocker } from '../../../../src/util/locking/GreedyReadWriteLocker';
import type { ResourceLocker } from '../../../../src/util/locking/ResourceLocker';
// A simple ResourceLocker that keeps a queue of lock requests
class MemoryLocker implements ResourceLocker {
private readonly locks: Record<string, (() => void)[]>;
public constructor() {
this.locks = {};
}
public async acquire(identifier: ResourceIdentifier): Promise<void> {
const { path } = identifier;
if (!this.locks[path]) {
this.locks[path] = [];
} else {
return new Promise((resolve): void => {
this.locks[path].push(resolve);
});
}
}
public async release(identifier: ResourceIdentifier): Promise<void> {
const { path } = identifier;
if (this.locks[path].length > 0) {
this.locks[path].shift()!();
} else {
// eslint-disable-next-line @typescript-eslint/no-dynamic-delete
delete this.locks[path];
}
}
}
describe('A GreedyReadWriteLocker', (): void => {
let sourceLocker: ResourceLocker;
let storage: KeyValueStorage<string, number>;
const resourceId = { path: 'http://test.com/resource' };
const resource2Id = { path: 'http://test.com/resource2' };
let locker: GreedyReadWriteLocker;
beforeEach(async(): Promise<void> => {
sourceLocker = new MemoryLocker();
storage = new Map<string, number>() as any;
locker = new GreedyReadWriteLocker(sourceLocker, storage);
});
it('does not block single read operations.', async(): Promise<void> => {
await expect(locker.withReadLock(resourceId, (): any => 5)).resolves.toBe(5);
});
it('does not block single write operations.', async(): Promise<void> => {
await expect(locker.withWriteLock(resourceId, (): any => 5)).resolves.toBe(5);
});
it('errors when trying to writeLock a count identifier.', async(): Promise<void> => {
await expect(locker.withWriteLock({ path: `http://test.com/foo.count` }, (): any => 5))
.rejects.toThrow(ForbiddenHttpError);
locker = new GreedyReadWriteLocker(sourceLocker, storage, { count: 'dummy', read: 'read', write: 'write' });
await expect(locker.withWriteLock({ path: `http://test.com/foo.dummy` }, (): any => 5))
.rejects.toThrow(ForbiddenHttpError);
});
it('errors if the read counter has an unexpected value.', async(): Promise<void> => {
storage.get = jest.fn().mockResolvedValue(0);
await expect(locker.withReadLock(resourceId, (): any => 5)).rejects.toThrow(InternalServerError);
});
it('does not block multiple read operations.', async(): Promise<void> => {
const order: string[] = [];
const emitter = new EventEmitter();
const unlocks = [ 0, 1, 2 ].map((num): any => new Promise((resolve): any => emitter.on(`release${num}`, resolve)));
const promises = [ 0, 1, 2 ].map((num): any => locker.withReadLock(resourceId, async(): Promise<number> => {
order.push(`start ${num}`);
await unlocks[num];
order.push(`finish ${num}`);
return num;
}));
// Allow time to attach listeners
await new Promise(setImmediate);
emitter.emit('release2');
await expect(promises[2]).resolves.toBe(2);
emitter.emit('release0');
await expect(promises[0]).resolves.toBe(0);
emitter.emit('release1');
await expect(promises[1]).resolves.toBe(1);
expect(order).toEqual([ 'start 0', 'start 1', 'start 2', 'finish 2', 'finish 0', 'finish 1' ]);
});
it('blocks multiple write operations.', async(): Promise<void> => {
// Previous test but with write locks
const order: string[] = [];
const emitter = new EventEmitter();
const unlocks = [ 0, 1, 2 ].map((num): any => new Promise((resolve): any => emitter.on(`release${num}`, resolve)));
const promises = [ 0, 1, 2 ].map((num): any => locker.withWriteLock(resourceId, async(): Promise<number> => {
order.push(`start ${num}`);
await unlocks[num];
order.push(`finish ${num}`);
return num;
}));
// Allow time to attach listeners
await new Promise(setImmediate);
emitter.emit('release2');
// Allow time to finish write 2
await new Promise(setImmediate);
emitter.emit('release0');
emitter.emit('release1');
await Promise.all([ promises[2], promises[0], promises[1] ]);
expect(order).toEqual([ 'start 0', 'finish 0', 'start 1', 'finish 1', 'start 2', 'finish 2' ]);
});
it('allows multiple write operations on different resources.', async(): Promise<void> => {
// Previous test but with write locks
const order: string[] = [];
const emitter = new EventEmitter();
const resources = [ resourceId, resource2Id ];
const unlocks = [ 0, 1 ].map((num): any => new Promise((resolve): any => emitter.on(`release${num}`, resolve)));
const promises = [ 0, 1 ].map((num): any => locker.withWriteLock(resources[num], async(): Promise<number> => {
order.push(`start ${num}`);
await unlocks[num];
order.push(`finish ${num}`);
return num;
}));
// Allow time to attach listeners
await new Promise(setImmediate);
emitter.emit('release1');
await expect(promises[1]).resolves.toBe(1);
emitter.emit('release0');
await expect(promises[0]).resolves.toBe(0);
expect(order).toEqual([ 'start 0', 'start 1', 'finish 1', 'finish 0' ]);
});
it('blocks write operations during read operations.', async(): Promise<void> => {
const order: string[] = [];
const emitter = new EventEmitter();
const promRead = new Promise((resolve): any => {
emitter.on('releaseRead', resolve);
});
// We want to make sure the write operation only starts while the read operation is busy
// Otherwise the internal write lock might not be acquired yet
const delayedLockWrite = new Promise<void>((resolve): void => {
emitter.on('readStarted', (): void => {
// eslint-disable-next-line @typescript-eslint/no-floating-promises
locker.withWriteLock(resourceId, (): any => {
order.push('write');
resolve();
});
});
});
const lockRead = locker.withReadLock(resourceId, async(): Promise<void> => {
emitter.emit('readStarted');
order.push('read start');
await promRead;
order.push('read finish');
});
// Allow time to attach listeners
await new Promise(setImmediate);
const promAll = Promise.all([ delayedLockWrite, lockRead ]);
emitter.emit('releaseRead');
await promAll;
expect(order).toEqual([ 'read start', 'read finish', 'write' ]);
});
it('allows write operations on different resources during read operations.', async(): Promise<void> => {
const order: string[] = [];
const emitter = new EventEmitter();
const promRead = new Promise((resolve): any => {
emitter.on('releaseRead', resolve);
});
const delayedLockWrite = new Promise<void>((resolve): void => {
emitter.on('readStarted', (): void => {
// eslint-disable-next-line @typescript-eslint/no-floating-promises
locker.withWriteLock(resource2Id, (): any => {
order.push('write');
resolve();
});
});
});
const lockRead = locker.withReadLock(resourceId, async(): Promise<void> => {
emitter.emit('readStarted');
order.push('read start');
await promRead;
order.push('read finish');
});
// Allow time to attach listeners
await new Promise(setImmediate);
const promAll = Promise.all([ delayedLockWrite, lockRead ]);
emitter.emit('releaseRead');
await promAll;
expect(order).toEqual([ 'read start', 'write', 'read finish' ]);
});
it('prioritizes read operations when a read operation is waiting.', async(): Promise<void> => {
// This test is very similar to the previous ones but adds an extra read lock
const order: string[] = [];
const emitter = new EventEmitter();
const promRead1 = new Promise((resolve): any => emitter.on('releaseRead1', resolve));
const promRead2 = new Promise((resolve): any => emitter.on('releaseRead2', resolve));
const delayedLockWrite = new Promise<void>((resolve): void => {
emitter.on('readStarted', (): void => {
// eslint-disable-next-line @typescript-eslint/no-floating-promises
locker.withWriteLock(resourceId, (): any => {
order.push('write');
resolve();
});
});
});
const delayedLockRead2 = new Promise<void>((resolve): void => {
emitter.on('readStarted', (): void => {
// eslint-disable-next-line @typescript-eslint/no-floating-promises
locker.withReadLock(resourceId, async(): Promise<void> => {
order.push('read 2 start');
await promRead2;
order.push('read 2 finish');
resolve();
});
});
});
const lockRead = locker.withReadLock(resourceId, async(): Promise<void> => {
emitter.emit('readStarted');
order.push('read 1 start');
await promRead1;
order.push('read 1 finish');
});
// Allow time to attach listeners
await new Promise(setImmediate);
const promAll = Promise.all([ delayedLockWrite, lockRead, delayedLockRead2 ]);
emitter.emit('releaseRead1');
// Allow time to finish read 1
await new Promise(setImmediate);
emitter.emit('releaseRead2');
await promAll;
expect(order).toEqual([ 'read 1 start', 'read 2 start', 'read 1 finish', 'read 2 finish', 'write' ]);
});
it('blocks read operations during write operations.', async(): Promise<void> => {
// Again similar but with read and write order switched
const order: string[] = [];
const emitter = new EventEmitter();
const promWrite = new Promise((resolve): any => {
emitter.on('releaseWrite', resolve);
});
// We want to make sure the read operation only starts while the write operation is busy
const delayedLockRead = new Promise<void>((resolve): void => {
emitter.on('writeStarted', (): void => {
// eslint-disable-next-line @typescript-eslint/no-floating-promises
locker.withReadLock(resourceId, (): any => {
order.push('read');
resolve();
});
});
});
const lockWrite = locker.withWriteLock(resourceId, async(): Promise<void> => {
emitter.emit('writeStarted');
order.push('write start');
await promWrite;
order.push('write finish');
});
// Allow time to attach listeners
await new Promise(setImmediate);
const promAll = Promise.all([ delayedLockRead, lockWrite ]);
emitter.emit('releaseWrite');
await promAll;
expect(order).toEqual([ 'write start', 'write finish', 'read' ]);
});
});