102 Commits

Author SHA1 Message Date
Joachim Van Herwegen
c9e4c7041c refactor: Prevent for-each calls 2023-11-02 09:49:17 +01:00
Joachim Van Herwegen
990184dbb5 refactor: Use node protocol when importing builtins 2023-11-02 09:49:17 +01:00
Joachim Van Herwegen
def0b5c732 refactor: Enable consistent newlines in lists 2023-11-02 09:49:17 +01:00
Joachim Van Herwegen
3bb3004abb refactor: Bring lint config back to original strictness 2023-11-02 09:49:17 +01:00
Joachim Van Herwegen
6248ed0938 refactor: Replace linting configurations
The previous package was outdated, preventing us from updating TS.
This one also lints YAML and JSON,
and applies many more rules to the test files,
explaining all the changes in this PR.
2023-11-02 09:49:17 +01:00
Joachim Van Herwegen
0ac7d407bf feat: Add migration for v6 account data 2023-10-06 12:54:26 +02:00
Joachim Van Herwegen
307dba3219 fix: Rename cookie field to authorization 2023-10-06 11:05:00 +02:00
Joachim Van Herwegen
4230db5038 feat: Use IndexedStorage to store account data 2023-10-06 11:04:58 +02:00
Joachim Van Herwegen
a47f5236ef feat: Full rework of account management
Complete rewrite of the account management and related systems.
Makes the architecture more modular,
allowing for easier extensions and configurations.
2023-10-06 11:04:40 +02:00
Joachim Van Herwegen
531c299c7b refactor: Rename WebHook to Webhook 2023-10-06 08:39:42 +02:00
Joachim Van Herwegen
b0d7342b39 Merge branch 'main' into versions/next-major 2023-10-06 08:11:21 +02:00
Joachim Van Herwegen
9bf734817d test: Remove test tmp folder after all tests are finished 2023-10-06 08:10:28 +02:00
Joachim Van Herwegen
661357c985 chore: Remove eslint rule about deleting dynamic keys 2023-09-26 09:42:39 +02:00
Joachim Van Herwegen
6fc3f2cf4f test: Clear test data folder before running tests 2023-07-20 17:10:14 +02:00
Joachim Van Herwegen
c0a881b980 fix: Use correct type for Webhook notifications 2023-06-13 08:14:11 +02:00
Joachim Van Herwegen
d59a1595d5 feat: Replace WebHookSubscription2021 with WebHookChannel2023 2023-04-24 11:00:54 +02:00
Joachim Van Herwegen
134237a80f feat: Support Add/Remove notifications on containers 2023-04-20 15:42:04 +02:00
Joachim Van Herwegen
702e8f5f59 feat: Replace WebSocketSubscription2021 with WebSocketChannel2023 2023-04-20 14:32:13 +02:00
Joachim Van Herwegen
c36f15e2da feat: Generalize and extend notification channel type behaviour 2023-04-20 14:32:13 +02:00
Joachim Van Herwegen
8d31233075 refactor: Rename NotificationChannelInfo to NotificationChannel 2023-04-20 14:32:13 +02:00
Joachim Van Herwegen
61f04487a1 refactor: Rename subscription to notification channel 2023-04-20 14:32:13 +02:00
Joachim Van Herwegen
7c343a5fcc feat: Update notification object to match the updated examples 2023-04-20 14:32:13 +02:00
Joachim Van Herwegen
7cc0e3fbcc Merge branch 'main' into versions/6.0.0
# Conflicts:
#	src/init/ServerInitializer.ts
#	src/server/BaseHttpServerFactory.ts
#	src/server/HttpServerFactory.ts
#	src/server/WebSocketServerFactory.ts
#	test/unit/server/BaseHttpServerFactory.test.ts
2023-02-01 10:13:04 +01:00
Joachim Van Herwegen
c46d01d3d7 feat: Remove agent/user permission differentiation
This was only used for the WAC-Allow header and greatly simplifies how we use permissions.
2022-11-18 16:08:06 +01:00
Koen Luyten
bf0e35be37 feat: allow server to bind to Unix Domain Sockets 2022-11-18 15:41:20 +01:00
Joachim Van Herwegen
f54c34d1e0 feat: Add support for WebHookSubscription2021 2022-11-15 15:50:05 +01:00
Joachim Van Herwegen
b1f7a6a8b1 feat: Add support for WebSocketSubscription2021 2022-11-09 09:10:00 +01:00
Joachim Van Herwegen
4223dcf8a4 feat: Split up server creation and request handling
This allows us to decouple the WebSocket listening from the HTTP configs,
making these features completely orthogonal.
2022-11-09 09:10:00 +01:00
Joachim Van Herwegen
9f04e28dc6 chore: Clean up ACP imports 2022-11-02 09:17:35 +01:00
Joachim Van Herwegen
40f2c8ea42 feat: Update templates and generators to support ACP 2022-10-06 09:36:04 +02:00
Joachim Van Herwegen
728617ac77 test: Add ACP integration tests 2022-10-06 09:36:04 +02:00
Joachim Van Herwegen
21b2850696 fix: Remove MS-Author-Via header 2022-08-05 15:58:29 +02:00
Wout Slabbinck
ca62511d12
feat: Edit metadata resources (#1188)
* feat: only allow metadata to be edited with PATCH request & only allow metadata files to be edited when a resource is available

* fix: remove unnecesary log at POST

* feat: PUT resets metadata contents + not possible to add metadata with PUT to container

* feat: add metadataStrategy (auxiliaryStrategy) + use that strategy in operationhandlers

* feat: PUT request on existing LDPC is not allowed as it would be possible to edit (read reset) metadata

* test: add unit tests to operationhandlers to handle metadata editing

* test: add unit tests to representationPatchHandler to handle metadata editing

* fix: update dependency of meta.json to version 3.0.0

* fix: lint and dependency still v2

* fix: replaced file references to resource references + moved Patch check to new patchhandler which is more generic

* fix: moved checking metadata resources checking from DELETE and POST handler down to DABS

* fix: remove PATCH message about metadata extension

* fix: move PATCH message about metadata extension

* WIP: adding writeMetadata + getMetadata in DABS and add writeMetadata to DataAccessors (part 1)

* WIP: implement writeMetadata in memorybackend + change resourceExists

* WIP: implement writeMetadata in SparqlDataAccessor.ts

* test: fix test interfaces

* test: InMemoryDataAccessor.ts resulted into changing identifier for writeMetadata in DataAccessor.ts (now taking subject identifier instead of metadata resource identifier)

* test: accessor tests implemented for metadata

* test: add RdfImmutableCheckPatcher.test.ts

* test: add tests in DataAccessorBasedStore.test.ts

* test: fix template config for DynamicPods test

* test: add integration tests for metadata

* fix: change metaStrategy to metadataStrategy

* refactor: comments updated to new location CSS on github + some alphabetical edits

* refactor: remove getMetadata function in DABS as it is only used once

* refactor: add DataAccessorBasedStoreArgs to DataAccessorBasedStore.ts

* docs: modify documentation for writeMetadata function in DataAccessor.ts

* feat: ldp:contains is also part of the metadata resource of a container

* refactor: change function name and move check to DataAccessorBasedStore

* fix: fix tests for DABS and PutOperationHandler

* feat: avoid cloneRepresentation by introducing RdfPatcher, RdfStorePatcher and modifying ImmutableMetadataPatcher, N3Patcher, patching.json and SparqlUpdatePatcher

* test: fix patcher tests

* feat: create sparqlInsertMetadata in SparqlDataAccessor.ts

* fix: move check during put on container if it exists already back to PutOperationHandler.ts after discussion in PR

* test: update tests PutOperationHandler.ts and DataAccessorBasedStore.ts regarding previous commit

* test: add converter to DABS and replace rejection on data during container creation to warning

* test: implemented RdfPatcher test

* feat: remove ContainerPatcher

* fix: fix lint

* fix: fix integration tests

* refactor: fix minor issues mentioned in the PR

* WIP: problem with removeResponseMetadata

* refactor: remove responseMetadata in QuadToRdfConverter.ts

* feat: handle ResponeMetadata when writing to the store via a patch

* refactor: refactor based on comments in PR

* feat: make ImmutableMetadataPatcher.ts instantiation more clear

* test: achieve 100% coverage again

* fix: fix lint

* refactor: return to explicit arguments for the DABS

* fix: return to explicit arguments for the DABS (missed one)

* feat: optimise immutable checker

* fix: fix, enhance docs + optimise config files

* fix: DABS + QuadToRdfConverter feedback implemented

* fix: patching feedback implemented

* test: update operationhandler tests

* test: update integration tests after feedback

* test: update DABS tests after feedback

* test: update ImmutableMetadataPatcher.test.ts after feedback

* test: update patch tests after feedback

* docs: add documentation about editing metadata

* fix: config: intendation + name change + extra filters | filter pattern

* docs: tsdoc added to RdfStorePatcher.ts

* fix: DABS split implemented for getRepresentation + comment refactoring

* docs: further documentation on removing response data on serialization

* fix: DABS getRepresentation method

* docs: apply feedback from Joachim on the documentation of metadata-editing.md

* fix: indentation fix + fix metadata-editing.md documentation after feedback from Joachim

* docs: small fix in metadata-editing.md documentation after feedback from Joachim

* fix: fix metadata-editing.md documentation after feedback from Joachim

* fix: fix tests meta-editing after feedback Joachim

* feat: first attempt at RELEASE_NOTES.md

* docs: update release notes based on feedback

* docs: fix newline

* fix: patching config changes after feedback

* docs: metadata editing documentation changes after feedback

* docs: metadata editing documentation changes after feedback

* docs: metadata editing documentation changes after feedback

* feat: optimisation on ImmutableMetadataPatcher.ts algorithm

* feat: remove converter from DABS and add conversion for metadata resources in the RCS

* fix: Fix documentation RepresentationPatchHandler.ts + fix response graph not being stored due to convertingstore

* feat: make RepresentationPatcher generic

* test: generic RepresentationPatcher tests

* test: 100% coverage for patchers again

* feat: containers can be created with POST with no content-type

* feat: Immutable checks always with subject identifier

* feat: create AuxiliaryLinkMetadataWriter for adding description resources Link Header

* test: add tests for AuxiliaryLinkMetadataWriter and update them for ImmutableMetadataPatcher

* feat: remove metadataGenerator from acl.json and fix tests accordingly

* WIP: preserve metadata on PUT

* feat: preserve metadata on PUT

* fix: keep metadata on PATCHes

* test: add unit tests for preserving metadata on PUT

* fix: remove inConverter from sparql endpoint as that is already the default in the (converting.json)

* fix: add metadatastrategy to RepresentationConvertingStore in regex.json

* test: add integration tests for preserving metadata on PUT

* docs: update release notes and adding documentation about preserving metadata on PUT

* WIP: Template create setRepresentation

* fix: Move container exists and not allowed check to setRepresentation

* test: fix lint

* fix: update configs and documentation

* refactor: update and add documentation + small refactoring

* refactor: update and add documentation + small refactoring + fix tests

* fix: Dynamic pod config + tests

* fix: TemplatedResourcesGenerator does not create containers when they already exist

* fix: metadata preservation now deals with complex content types

* docs: explain the case when there is no content-type

* fix: minor comments
2022-08-04 15:24:36 +02:00
Joachim Van Herwegen
7085252b3f feat: Update ModesExtractors to support new permission interface 2022-07-11 14:07:43 +02:00
Wannes Kerckhove
1c65b06392 fix: Introducing initializers for handling lock cleanup on start 2022-07-06 10:00:15 +02:00
Wannes Kerckhove
ff80079000 feat: file-based backend fallback for unknown media types 2022-04-28 14:20:49 +02:00
Wannes Kerckhove
dbdb9b424e fix: %2F not handled correctly in file backend #1184
Fix
2022-04-21 15:17:32 +02:00
Joachim Van Herwegen
3d6e3d2e39 chore: Drop support for Node 12 2022-04-21 14:54:12 +02:00
Thomas Dupont
e2e2d0864f feat: add a Redis based Read-Write Locker
refactor: more elegant way of providing default attemptSettings to constructor

style(jsdoc): rewording of jsdoc comment

fix: RegExp(/regex/) => /regex/

fix: Replace Error with InternalServerError

docs: jsdoc for RedisReadWriteLocker class

feat: make RedisReadWriteLocker a ResourceLocker too

test: coverage back to 100%

refactor: linting fix

style(jsdoc): Add explanation to tryRedisFn() method

refactor: remove RedisResourceLocker

fix: bug in lua script

chore(deps): update ioredis, remove redlock

refactor: removed RedisResourceLocker in favor of generic RedisLocker class

test: add redis lua scripts tests and integrate all 3 redis integration tests in 1

refactor: remove .vscode folder from index

refactor: Add some typing and  change redis references to Redis in comments

refactor: more changes after PR review

refactor: remove redis.json

refactor: rename redis-rw.json to redis.json

docs: added readme and release notes
2022-04-19 13:52:31 +02:00
Joachim Van Herwegen
e6519992bf Merge branch 'main' into versions/4.0.0
# Conflicts:
#	test/unit/storage/accessors/FileDataAccessor.test.ts
2022-04-15 10:28:50 +02:00
Joachim Van Herwegen
db906ae872 test: Add pod seeding integration test 2022-04-14 10:34:04 +02:00
Jasper Vaneessen
fe39f97ee0
refactor: Use fs-extra instead of fs to simplify file access
* refactor: use fs-extra instead of fs

* tests: manual mocks for fs-extra base + ensureDir

* refactor: mockFileSystem + mockFs and mockFsExtra

* add remove mock and some further test tweaks

* test: FileDataAccessor tests passing

* refactor: remove try-catch due to fs-extra handlin

* refactor: fs-extra in atomicFileDataAccessor

* refactor: AtomicFileDataAccessor fs-extra

* test: fix coverage

* refactor: use read/writeJson from fs-extra

* refactor: less duplicate mocking code

* refactor: re-use opendir mocking code
2022-04-12 11:02:30 +02:00
Joachim Van Herwegen
ce5a98b16f test: Add integration test that simulates running 2 hours 2022-03-31 17:00:09 +02:00
Joachim Van Herwegen
16e9368734 test: Add flushPromises utility function 2022-03-31 17:00:09 +02:00
Joachim Van Herwegen
6e98c6aae4 feat: Dynamically generate Allow and Accept-* headers 2022-03-29 14:52:36 +02:00
Joachim Van Herwegen
6f83ac5ead test: Create permission table to automate tests 2022-03-18 14:15:12 +01:00
Joachim Van Herwegen
90a6460c8d Merge branch 'main' into versions/3.0.0
# Conflicts:
#	package-lock.json
#	test/integration/Identity.test.ts
#	test/integration/RepresentationConverter.test.ts
2022-01-25 11:44:24 +01:00
Joachim Van Herwegen
a9941ebe78 feat: Add support for N3 Patch 2022-01-25 11:30:42 +01:00
Joachim Van Herwegen
af049124bf chore: Update eslint dependencies 2022-01-24 09:37:50 +01:00
Arthur Joppart
0cb4d7b161
feat: Add support for quota limits
* feat: implemented SizeReporter and FileSizeReporter

* test: FileSizeReporter tests

* feat: added QuotedDataAccessor

* test: added extra test to check recursiveness of filesizereporter

* feat: added QuotaStrategy interface

* feat: further progress in different files

* feat: wrote doc, tests and improved code

* feat: fixed bugs and code is now runnable and buildable

* feat: finished implementation

* fix: revert accidental chanegs

* fix: fileSizeReported did not count container size

* fix: bug calculating container sizes fixed

* test: FileSizeReporter tests

* test: QuotaDataValidator tests

* test: QuotaError tests

* fix: removed console.log

* doc: added doc to several files

* doc: changed doc for QuotaStrategy to new implementation

* fix: improved content length regex

* feat: improved GlobalQuotaStrategy code

* fix: made FileSizeReported readonly

* feat: added comments to quota-file.json

* fix: changed default tempFilePath variable

* test: included new tempFilePath variable in testing

* chore: created seperate command for start:file:quota to pass tests

* feat: removed all sync fs calls from FileSizeReporter

* feat: minor changes in multple files

* fix: changed function signatures to be in line with others

* feat: optimized quota data validation

* feat: improved FileSizeReporter code

* fix: corrected calculation of containersizes and fixed erroring edgecase

* feat: save content-length as number in metadata

* feat: added comments and changed GlobalQuotaStrategy constructor

* feat: changed file names and added small comment

* test: AtomicFileDataAccessor tests

* test: completed FileSizeReporter tests

* fix: content-length is now saved correctly in RepresentationMetadata

* feat: adapted content length metadata + tests

* fix: removed tempFilePath variable

* fix: reverted .gitignore

* fix: forgot to remove tempFilePath variable from componentsjs config

* test: GlobalQuotaStrategy tests

* feat: replaced DataValidator with Validator

* feat: reworked DataValidator

* feat: added calcultateChunkSize() to SizeReporter

* test: updated FileSizeReporter tests

* fix: tempFile location now relative to rootFilePath

* test: QuotaDataValidator tests

* fix: corrected FileSizeReporter tests

* fix: adapted FileSizeReporter tests

* fix: FileSizeReporter bug on Windows

* fix: regex linting error

* feat: changed Validator class

* feat: added PodQuotaStrategy to enable suota on a per pod basis

* chore: bump context versions

* fix: Capitalized comments in json file

* chore: renamed ValidatorArgs to ValidatorInput

* chore: order all exports

* fix: made TODO comment clearer

* chore: added seperated config files for global and pod based quota + fixed comments

* chore: made minor changes to comments

* feat: added PassthroughDataAccessor

* feat: added PasstroughtDataAccessor + tests

* fix: added invalid header check to ContentLengthParser

* chore: improved mocks

* chore: move quota limit higher up in config

* fix: atomicity issue in AtomicFileDataAccessor

* chore: moved .internal folder to config from FileSizeReporter

* fix: improved algorithm to ignore folders while calculating file size in FileSizeReporter

* fix: changes to support containers in the future

* fix: added error handling to prevent reading of unexistent files

* feat: added generic type to SizeReporter to calculate chunk sizes

* test: use mocked DataAccessor

* chore: added some comments to test and made minor improvement

* fix: fs mock rename

* chore: QuotaStrategy.estimateSize refactor

* chore: move trackAvailableSpace to abstract class QuotaStrategy

* fix: improved test case

* test: quota integration tests

* chore: edited some comments

* chore: change lstat to stat

* feat: moved estimateSize to SizeReporter to be consistent with calcultateChunkSize

* test: finish up tests to reach coverage

* fix: basic config

* fix: minor changes to test CI run

* fix: small fix for windows

* fix: improved writing to file

* chore: linting errors

* chore: rename trackAvailableSpace

* test: improved integration tests

* test: logging info for test debugging

* test: extra logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: improved Quota integration test setup

* test: improve quota tests for CI run

* test: debugging Quota test

* test: uncommented global quota test

* test: changed global quota parameters

* test: logging for debugging

* test: logging cleanup

* chore: minor changes, mostly typo fixes

* chore: remove console.log

* fix: getting inconsistent results

* chore: try fix index.ts CI error

* chore: try fix CI error

* chore: try fix CI error

* chore: revert last commits

* chore: fix inconsistent files with origin

* test: minor test improvements

* chore: minor refactors and improvements

* fix: added extra try catch for breaking bug

* chore: improve config

* chore: minor code improvements

* test: use mockFs

* feat: add extra check in podQuotaStrategy

* chore: replace handle by handleSafe in ValidatingDataAccessor

* chore: typo

* test: improved Quota integration tests

* test: made comment in test more correct

* fix: rm -> rmdir for backwards compatibility

* fix: fsPromises issue

* chore: leave out irrelevant config

* chore: removed start script from package.json

* fix: Small fixes

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2022-01-21 10:49:05 +01:00