269 Commits

Author SHA1 Message Date
Joachim Van Herwegen
e1af8ee66e fix: Fix incorrect config import 2022-10-13 15:06:50 +02:00
Joachim Van Herwegen
f7742cffef fix: Add missing parameter to sparql-file-storage.json configuration 2022-10-05 15:43:58 +02:00
Joachim Van Herwegen
e050f8be93 feat: Allow multiple configurations to be used during startup 2022-09-26 10:26:00 +02:00
Jasper Vaneessen
9a5fc674f3
style: Enforce linting rules on markdown files
* chore: add markdownlint-cli2 and config for mkdocs

* style: enforce linting rules on mkdocs md files

* chore: tweaks to markdownlint rules

* style: linting changelog

* style: linting release notes

* style: linting .github md files

* style: further linting of docs

* style: linting readmes

* chore: update linting script entries

* docs: tweak release after rebase

* chore: simplify root md linting config

* chore: extend base config

* chore: implement requested changes

* chore: remove unnecessary exception

* chore: fix comment type

* styling: single config + list spacing

* chore: implement requested changes

* chore: use .cjs files for markdownlint config

* chore: implement requested changes
2022-08-25 11:32:09 +02:00
Joachim Van Herwegen
d73480bde4 chore(release): Update configs to v5.0.0 2022-08-08 10:47:30 +02:00
Joachim Van Herwegen
839a923df6 fix: Update the default timeout to 6s 2022-08-05 16:19:33 +02:00
Joachim Van Herwegen
21b2850696 fix: Remove MS-Author-Via header 2022-08-05 15:58:29 +02:00
Joachim Van Herwegen
740ba3398b Merge branch 'main' into versions/5.0.0
# Conflicts:
#	RELEASE_NOTES.md
#	config/app/variables/cli/cli.json
#	config/app/variables/default.json
#	package-lock.json
#	package.json
2022-08-04 17:18:34 +02:00
Wout Slabbinck
ca62511d12
feat: Edit metadata resources (#1188)
* feat: only allow metadata to be edited with PATCH request & only allow metadata files to be edited when a resource is available

* fix: remove unnecesary log at POST

* feat: PUT resets metadata contents + not possible to add metadata with PUT to container

* feat: add metadataStrategy (auxiliaryStrategy) + use that strategy in operationhandlers

* feat: PUT request on existing LDPC is not allowed as it would be possible to edit (read reset) metadata

* test: add unit tests to operationhandlers to handle metadata editing

* test: add unit tests to representationPatchHandler to handle metadata editing

* fix: update dependency of meta.json to version 3.0.0

* fix: lint and dependency still v2

* fix: replaced file references to resource references + moved Patch check to new patchhandler which is more generic

* fix: moved checking metadata resources checking from DELETE and POST handler down to DABS

* fix: remove PATCH message about metadata extension

* fix: move PATCH message about metadata extension

* WIP: adding writeMetadata + getMetadata in DABS and add writeMetadata to DataAccessors (part 1)

* WIP: implement writeMetadata in memorybackend + change resourceExists

* WIP: implement writeMetadata in SparqlDataAccessor.ts

* test: fix test interfaces

* test: InMemoryDataAccessor.ts resulted into changing identifier for writeMetadata in DataAccessor.ts (now taking subject identifier instead of metadata resource identifier)

* test: accessor tests implemented for metadata

* test: add RdfImmutableCheckPatcher.test.ts

* test: add tests in DataAccessorBasedStore.test.ts

* test: fix template config for DynamicPods test

* test: add integration tests for metadata

* fix: change metaStrategy to metadataStrategy

* refactor: comments updated to new location CSS on github + some alphabetical edits

* refactor: remove getMetadata function in DABS as it is only used once

* refactor: add DataAccessorBasedStoreArgs to DataAccessorBasedStore.ts

* docs: modify documentation for writeMetadata function in DataAccessor.ts

* feat: ldp:contains is also part of the metadata resource of a container

* refactor: change function name and move check to DataAccessorBasedStore

* fix: fix tests for DABS and PutOperationHandler

* feat: avoid cloneRepresentation by introducing RdfPatcher, RdfStorePatcher and modifying ImmutableMetadataPatcher, N3Patcher, patching.json and SparqlUpdatePatcher

* test: fix patcher tests

* feat: create sparqlInsertMetadata in SparqlDataAccessor.ts

* fix: move check during put on container if it exists already back to PutOperationHandler.ts after discussion in PR

* test: update tests PutOperationHandler.ts and DataAccessorBasedStore.ts regarding previous commit

* test: add converter to DABS and replace rejection on data during container creation to warning

* test: implemented RdfPatcher test

* feat: remove ContainerPatcher

* fix: fix lint

* fix: fix integration tests

* refactor: fix minor issues mentioned in the PR

* WIP: problem with removeResponseMetadata

* refactor: remove responseMetadata in QuadToRdfConverter.ts

* feat: handle ResponeMetadata when writing to the store via a patch

* refactor: refactor based on comments in PR

* feat: make ImmutableMetadataPatcher.ts instantiation more clear

* test: achieve 100% coverage again

* fix: fix lint

* refactor: return to explicit arguments for the DABS

* fix: return to explicit arguments for the DABS (missed one)

* feat: optimise immutable checker

* fix: fix, enhance docs + optimise config files

* fix: DABS + QuadToRdfConverter feedback implemented

* fix: patching feedback implemented

* test: update operationhandler tests

* test: update integration tests after feedback

* test: update DABS tests after feedback

* test: update ImmutableMetadataPatcher.test.ts after feedback

* test: update patch tests after feedback

* docs: add documentation about editing metadata

* fix: config: intendation + name change + extra filters | filter pattern

* docs: tsdoc added to RdfStorePatcher.ts

* fix: DABS split implemented for getRepresentation + comment refactoring

* docs: further documentation on removing response data on serialization

* fix: DABS getRepresentation method

* docs: apply feedback from Joachim on the documentation of metadata-editing.md

* fix: indentation fix + fix metadata-editing.md documentation after feedback from Joachim

* docs: small fix in metadata-editing.md documentation after feedback from Joachim

* fix: fix metadata-editing.md documentation after feedback from Joachim

* fix: fix tests meta-editing after feedback Joachim

* feat: first attempt at RELEASE_NOTES.md

* docs: update release notes based on feedback

* docs: fix newline

* fix: patching config changes after feedback

* docs: metadata editing documentation changes after feedback

* docs: metadata editing documentation changes after feedback

* docs: metadata editing documentation changes after feedback

* feat: optimisation on ImmutableMetadataPatcher.ts algorithm

* feat: remove converter from DABS and add conversion for metadata resources in the RCS

* fix: Fix documentation RepresentationPatchHandler.ts + fix response graph not being stored due to convertingstore

* feat: make RepresentationPatcher generic

* test: generic RepresentationPatcher tests

* test: 100% coverage for patchers again

* feat: containers can be created with POST with no content-type

* feat: Immutable checks always with subject identifier

* feat: create AuxiliaryLinkMetadataWriter for adding description resources Link Header

* test: add tests for AuxiliaryLinkMetadataWriter and update them for ImmutableMetadataPatcher

* feat: remove metadataGenerator from acl.json and fix tests accordingly

* WIP: preserve metadata on PUT

* feat: preserve metadata on PUT

* fix: keep metadata on PATCHes

* test: add unit tests for preserving metadata on PUT

* fix: remove inConverter from sparql endpoint as that is already the default in the (converting.json)

* fix: add metadatastrategy to RepresentationConvertingStore in regex.json

* test: add integration tests for preserving metadata on PUT

* docs: update release notes and adding documentation about preserving metadata on PUT

* WIP: Template create setRepresentation

* fix: Move container exists and not allowed check to setRepresentation

* test: fix lint

* fix: update configs and documentation

* refactor: update and add documentation + small refactoring

* refactor: update and add documentation + small refactoring + fix tests

* fix: Dynamic pod config + tests

* fix: TemplatedResourcesGenerator does not create containers when they already exist

* fix: metadata preservation now deals with complex content types

* docs: explain the case when there is no content-type

* fix: minor comments
2022-08-04 15:24:36 +02:00
Joachim Van Herwegen
9dcba1a288 refactor: Add imports that allow for HTTPS through CLI params 2022-07-28 11:19:24 +02:00
Joachim Van Herwegen
4e999eba98 feat: Enable strict parsing of CLI arguments 2022-07-28 10:58:57 +02:00
Joachim Van Herwegen
f609f1a9c5 feat: Accept both Settings and VariableBindings to create an App 2022-07-27 10:00:43 +02:00
Joachim Van Herwegen
d7d1480e8e refactor: Fix indentation 2022-07-26 12:01:21 +02:00
Joachim Van Herwegen
d5bcec704c feat: Update configs based on all permission changes 2022-07-11 14:07:43 +02:00
Thomas Dupont
a461586921 feat: args as env vars
refactor: use constant for env var prefix
2022-07-07 14:29:17 +02:00
Thomas Dupont
311f8756ec fix: return contenttype header value string with parameters 2022-07-07 10:02:24 +02:00
Wannes Kerckhove
1c65b06392 fix: Introducing initializers for handling lock cleanup on start 2022-07-06 10:00:15 +02:00
Joachim Van Herwegen
affcb7a7b3 Merge branch 'main' into versions/5.0.0
# Conflicts:
#	RELEASE_NOTES.md
#	config/quota-file.json
2022-07-04 13:53:17 +02:00
Joachim Van Herwegen
3dc8b497bb
chore: Add email to senderName for email sender 2022-06-28 15:25:11 +02:00
Joachim Van Herwegen
26b42f0b17 fix: Enable ACL in default quota config 2022-06-07 10:38:26 +02:00
Joachim Van Herwegen
3f817b14b0 feat: Extend OIDC error descriptions 2022-06-01 10:05:36 +02:00
Joachim Van Herwegen
df0825936a feat: Parse Accept headers as early as possible 2022-06-01 10:05:36 +02:00
Jasper Vaneessen
2814e72b34
feat: Respect root path for static assets and HTML links
* feat: rootpath for static assets and links1

* fix: static asset handler respects root path

* fix: use rootPath for links

* tests: fix the tests after adding consuctor params

* feat: change matchregex instead of stored URLs

* feat: baseUrl for handlebar engine and templates

* feat: full baseUrl passed to templates

* test: fix integration tests + templates

* chore: implement requested changes

* docs: Describe TemplateEngine interface changes

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2022-05-24 10:20:41 +02:00
Thomas Dupont
5399e75ae4 feat: regexRoutes stored as ordered array 2022-05-18 09:49:51 +02:00
Thomas Dupont
236bbc6e5d feat: initial proposal for multithreaded execution 2022-05-16 10:15:37 +02:00
Wannes Kerckhove
fbbccb0cf1 fix: Stop creating meta files for each new resource #1217 2022-05-06 08:20:55 +02:00
Joachim Van Herwegen
77eeae9c9e chore: Update configs to v5.0.0 2022-05-05 11:20:53 +02:00
Thomas Dupont
fa78bc6856 feat: add a process-/thread-safe file-based ResourceLocker
test: unit test succeeds

fix: not quiting loop when releasing unexisting lock

refactor: pull wait() function into TimerUtils

feat: store all locks inside a single lock folder

feat: use md5 hashing for filepath hashes

test: coverage back to 100%

fix: store locks in proper .internal/locks folder
feat: reworked tryfn

test: coverage back to 100%

buidl: package json types next to lib

style: linting

dos: add more documentation to Locker classes

refactor: SingleThreadedResourceLocker -> MemoryResourceLocker

refactor: MultiThreadedResourceLocker -> FileSystemResourceLocker

feat: update all file-based backend configs to use the new FileSystemResourceLocker

feat: add warning on starting the MemoryResourceLocker in a worker process

test: coverage back to 100%

fix: finalizer of file.json was configured wrong

docs: updated release notes for 5.0.0

refactor: incorporated changes so far

refactor: retryFunctions are less complex now

test: jitter fix
2022-04-28 14:12:30 +02:00
Joachim Van Herwegen
d2bc995272 refactor: Replace RedirectAllHttpHandler usage with RedirectingHttpHandler 2022-04-25 09:09:39 +02:00
Joachim Van Herwegen
6f4e70dbb9 fix: Change YargsCliExtractor structure to avoid Components.js issues 2022-04-25 09:09:39 +02:00
Joachim Van Herwegen
b84788e05f refactor: Remove unnecessary Components.js workarounds 2022-04-25 09:09:39 +02:00
Joachim Van Herwegen
225affe42c docs: Update config docs 2022-04-25 09:09:39 +02:00
Joachim Van Herwegen
cfdd1221f0 fix: Always define @type in configs 2022-04-25 09:09:39 +02:00
Joachim Van Herwegen
ce27bec207 fix: Update prefixes in all configs 2022-04-25 09:09:39 +02:00
Joachim Van Herwegen
784bfc8c21 chore: Update configs to v4.0.0 2022-04-19 15:13:55 +02:00
Thomas Dupont
e2e2d0864f feat: add a Redis based Read-Write Locker
refactor: more elegant way of providing default attemptSettings to constructor

style(jsdoc): rewording of jsdoc comment

fix: RegExp(/regex/) => /regex/

fix: Replace Error with InternalServerError

docs: jsdoc for RedisReadWriteLocker class

feat: make RedisReadWriteLocker a ResourceLocker too

test: coverage back to 100%

refactor: linting fix

style(jsdoc): Add explanation to tryRedisFn() method

refactor: remove RedisResourceLocker

fix: bug in lua script

chore(deps): update ioredis, remove redlock

refactor: removed RedisResourceLocker in favor of generic RedisLocker class

test: add redis lua scripts tests and integrate all 3 redis integration tests in 1

refactor: remove .vscode folder from index

refactor: Add some typing and  change redis references to Redis in comments

refactor: more changes after PR review

refactor: remove redis.json

refactor: rename redis-rw.json to redis.json

docs: added readme and release notes
2022-04-19 13:52:31 +02:00
Joachim Van Herwegen
570e167a36 fix: Keep storage paths consistent with previous version 2022-04-19 11:52:56 +02:00
Joachim Van Herwegen
2ec8fabd54 feat: Add support for client_credentials authentication 2022-04-15 11:20:26 +02:00
Joachim Van Herwegen
bedab907f9 feat: Allow dynamically adding CLI parameters in configs 2022-04-15 10:56:05 +02:00
Joachim Van Herwegen
e6519992bf Merge branch 'main' into versions/4.0.0
# Conflicts:
#	test/unit/storage/accessors/FileDataAccessor.test.ts
2022-04-15 10:28:50 +02:00
Joachim Van Herwegen
30799f6667 fix: Add missing imports from default configs 2022-04-15 10:15:58 +02:00
Joachim Van Herwegen
1b7cc1ea3a Merge branch 'main' into versions/4.0.0
# Conflicts:
#	test/unit/util/errors/RedirectHttpError.test.ts
2022-04-11 10:21:49 +02:00
Wouter Termont
468e11d906
feat: Add RedirectingHttpHandler
* feat: redirect handler

Signed-off-by: Wouter Termont <woutermont@gmail.com>

* chore: rewrite reduction as loop

Signed-off-by: Wouter Termont <woutermont@gmail.com>

* chore: remove example import

Signed-off-by: Wouter Termont <woutermont@gmail.com>

* feat: add more redirect flavours

Signed-off-by: Wouter Termont <woutermont@gmail.com>

* chore: RedirectingHttpHandler with RedirectAllHttpHandler

Signed-off-by: Wouter Termont <woutermont@gmail.com>

* chore: replace RedirectAllHttpHandler with RedirectingHttpHandler

* chore: revert 5956385 (chore: replace RedirectAllHttpHandler with RedirectingHttpHandler)

This reverts commit 5956385c4180e8e8914b9199c4ed6ca8ae7d39fb.

* docs: complete constructor params

Signed-off-by: Wouter Termont <woutermont@gmail.com>
2022-04-11 09:49:36 +02:00
wkerckho
d42125a91d
fix: Throw error when accessing URLs out of scope
* feat: allow OriginalUrlExtractor to be configured with an identifierStrategy. Included the existing parameter 'includeQueryString' in the OriginalUrlExtractorArgs type.

* test: fixed OriginalUrlExtractor instantiation in OriginalUrlExtractor and RequestParser tests

* fix: Server no longer warns when accessing a URL out of scope #1148

* test: updated tests broken by #1148 fix

* test: ensuring 100% coverage

Co-authored-by: Wannes Kerckhove <wannes.kerckhove@ugent.be>
2022-04-08 09:33:22 +02:00
Joachim Van Herwegen
97e600bf4f fix: Undo authorization on OPTIONS requests 2022-04-04 13:35:15 +02:00
Thomas Dupont
48efc6fae1
feat: Create MetadataParser that detects JSON with Context link and throws an error
* feat: add PlainJsonLdFilter to reject JSON with context link

* refactor: abstract parseLinkHeader into HeaderUtils

* docs: typo in comment field

Co-authored-by: Ruben Verborgh <ruben@verborgh.org>

* refactor: Replace BadRequestHttpError with NotImplementedError

Co-authored-by: Ruben Verborgh <ruben@verborgh.org>

* refactor: incorporate requested changes

* refactor: requested changes incorporated

* refactor: remove obsolete code lines

Co-authored-by: Ruben Verborgh <ruben@verborgh.org>
2022-04-01 14:25:09 +02:00
Joachim Van Herwegen
6e98c6aae4 feat: Dynamically generate Allow and Accept-* headers 2022-03-29 14:52:36 +02:00
Joachim Van Herwegen
30ad3015f0 refactor: Move key/value storages to relevant configs 2022-03-28 11:43:14 +02:00
Joachim Van Herwegen
a1a6ce01fa feat: Create separate storage to generate keys 2022-03-28 11:43:14 +02:00
Joachim Van Herwegen
ad3edcf1a8 feat: Handle OPTIONS requests in OperationHandler 2022-03-18 14:15:12 +01:00