46 Commits

Author SHA1 Message Date
Joachim Van Herwegen
01623e0b5c
docs: Explain storage/location import options
* docs: Explain storage/location import options

* docs: Fix language

Co-authored-by: Ted Thibodeau Jr <tthibodeau@openlinksw.com>

---------

Co-authored-by: Ted Thibodeau Jr <tthibodeau@openlinksw.com>
2023-11-17 11:26:54 +01:00
Joachim Van Herwegen
6248ed0938 refactor: Replace linting configurations
The previous package was outdated, preventing us from updating TS.
This one also lints YAML and JSON,
and applies many more rules to the test files,
explaining all the changes in this PR.
2023-11-02 09:49:17 +01:00
Joachim Van Herwegen
6eaaa42015 chore(release): Update configs to v7.0.0 2023-10-19 09:13:48 +02:00
Joachim Van Herwegen
b5a61cbb08 feat: Use new MaxKeyLengthStorage to prevent keys that are too long 2023-10-11 12:47:02 +02:00
Joachim Van Herwegen
e1c5189cb8 feat: Remove base64 encoding from storages 2023-10-11 12:47:02 +02:00
Joachim Van Herwegen
ade977bb4f feat: Move storage location decision to separate import 2023-10-06 10:11:59 +02:00
Joachim Van Herwegen
c95198285c Merge branch 'main' into versions/next-major 2023-10-05 14:28:06 +02:00
Joachim Van Herwegen
3e9adef4cf feat: Add support for range headers 2023-10-05 10:07:49 +02:00
Joachim Van Herwegen
154d981684 feat: Split up EncodingPathStorage functionality into different classes 2023-09-20 11:13:56 +02:00
Joachim Van Herwegen
d9c0e9b043 chore(release): Update configs to v6.0.0 2023-02-01 10:54:37 +01:00
Joachim Van Herwegen
0d6b895df3 feat: Hash lock-related identifiers 2023-02-01 09:47:13 +01:00
Joachim Van Herwegen
7422fbffe7 feat: Take preferences as input in RepresentationConvertingStore 2023-01-11 16:03:30 +01:00
Jasper Vaneessen
9a5fc674f3
style: Enforce linting rules on markdown files
* chore: add markdownlint-cli2 and config for mkdocs

* style: enforce linting rules on mkdocs md files

* chore: tweaks to markdownlint rules

* style: linting changelog

* style: linting release notes

* style: linting .github md files

* style: further linting of docs

* style: linting readmes

* chore: update linting script entries

* docs: tweak release after rebase

* chore: simplify root md linting config

* chore: extend base config

* chore: implement requested changes

* chore: remove unnecessary exception

* chore: fix comment type

* styling: single config + list spacing

* chore: implement requested changes

* chore: use .cjs files for markdownlint config

* chore: implement requested changes
2022-08-25 11:32:09 +02:00
Wout Slabbinck
ca62511d12
feat: Edit metadata resources (#1188)
* feat: only allow metadata to be edited with PATCH request & only allow metadata files to be edited when a resource is available

* fix: remove unnecesary log at POST

* feat: PUT resets metadata contents + not possible to add metadata with PUT to container

* feat: add metadataStrategy (auxiliaryStrategy) + use that strategy in operationhandlers

* feat: PUT request on existing LDPC is not allowed as it would be possible to edit (read reset) metadata

* test: add unit tests to operationhandlers to handle metadata editing

* test: add unit tests to representationPatchHandler to handle metadata editing

* fix: update dependency of meta.json to version 3.0.0

* fix: lint and dependency still v2

* fix: replaced file references to resource references + moved Patch check to new patchhandler which is more generic

* fix: moved checking metadata resources checking from DELETE and POST handler down to DABS

* fix: remove PATCH message about metadata extension

* fix: move PATCH message about metadata extension

* WIP: adding writeMetadata + getMetadata in DABS and add writeMetadata to DataAccessors (part 1)

* WIP: implement writeMetadata in memorybackend + change resourceExists

* WIP: implement writeMetadata in SparqlDataAccessor.ts

* test: fix test interfaces

* test: InMemoryDataAccessor.ts resulted into changing identifier for writeMetadata in DataAccessor.ts (now taking subject identifier instead of metadata resource identifier)

* test: accessor tests implemented for metadata

* test: add RdfImmutableCheckPatcher.test.ts

* test: add tests in DataAccessorBasedStore.test.ts

* test: fix template config for DynamicPods test

* test: add integration tests for metadata

* fix: change metaStrategy to metadataStrategy

* refactor: comments updated to new location CSS on github + some alphabetical edits

* refactor: remove getMetadata function in DABS as it is only used once

* refactor: add DataAccessorBasedStoreArgs to DataAccessorBasedStore.ts

* docs: modify documentation for writeMetadata function in DataAccessor.ts

* feat: ldp:contains is also part of the metadata resource of a container

* refactor: change function name and move check to DataAccessorBasedStore

* fix: fix tests for DABS and PutOperationHandler

* feat: avoid cloneRepresentation by introducing RdfPatcher, RdfStorePatcher and modifying ImmutableMetadataPatcher, N3Patcher, patching.json and SparqlUpdatePatcher

* test: fix patcher tests

* feat: create sparqlInsertMetadata in SparqlDataAccessor.ts

* fix: move check during put on container if it exists already back to PutOperationHandler.ts after discussion in PR

* test: update tests PutOperationHandler.ts and DataAccessorBasedStore.ts regarding previous commit

* test: add converter to DABS and replace rejection on data during container creation to warning

* test: implemented RdfPatcher test

* feat: remove ContainerPatcher

* fix: fix lint

* fix: fix integration tests

* refactor: fix minor issues mentioned in the PR

* WIP: problem with removeResponseMetadata

* refactor: remove responseMetadata in QuadToRdfConverter.ts

* feat: handle ResponeMetadata when writing to the store via a patch

* refactor: refactor based on comments in PR

* feat: make ImmutableMetadataPatcher.ts instantiation more clear

* test: achieve 100% coverage again

* fix: fix lint

* refactor: return to explicit arguments for the DABS

* fix: return to explicit arguments for the DABS (missed one)

* feat: optimise immutable checker

* fix: fix, enhance docs + optimise config files

* fix: DABS + QuadToRdfConverter feedback implemented

* fix: patching feedback implemented

* test: update operationhandler tests

* test: update integration tests after feedback

* test: update DABS tests after feedback

* test: update ImmutableMetadataPatcher.test.ts after feedback

* test: update patch tests after feedback

* docs: add documentation about editing metadata

* fix: config: intendation + name change + extra filters | filter pattern

* docs: tsdoc added to RdfStorePatcher.ts

* fix: DABS split implemented for getRepresentation + comment refactoring

* docs: further documentation on removing response data on serialization

* fix: DABS getRepresentation method

* docs: apply feedback from Joachim on the documentation of metadata-editing.md

* fix: indentation fix + fix metadata-editing.md documentation after feedback from Joachim

* docs: small fix in metadata-editing.md documentation after feedback from Joachim

* fix: fix metadata-editing.md documentation after feedback from Joachim

* fix: fix tests meta-editing after feedback Joachim

* feat: first attempt at RELEASE_NOTES.md

* docs: update release notes based on feedback

* docs: fix newline

* fix: patching config changes after feedback

* docs: metadata editing documentation changes after feedback

* docs: metadata editing documentation changes after feedback

* docs: metadata editing documentation changes after feedback

* feat: optimisation on ImmutableMetadataPatcher.ts algorithm

* feat: remove converter from DABS and add conversion for metadata resources in the RCS

* fix: Fix documentation RepresentationPatchHandler.ts + fix response graph not being stored due to convertingstore

* feat: make RepresentationPatcher generic

* test: generic RepresentationPatcher tests

* test: 100% coverage for patchers again

* feat: containers can be created with POST with no content-type

* feat: Immutable checks always with subject identifier

* feat: create AuxiliaryLinkMetadataWriter for adding description resources Link Header

* test: add tests for AuxiliaryLinkMetadataWriter and update them for ImmutableMetadataPatcher

* feat: remove metadataGenerator from acl.json and fix tests accordingly

* WIP: preserve metadata on PUT

* feat: preserve metadata on PUT

* fix: keep metadata on PATCHes

* test: add unit tests for preserving metadata on PUT

* fix: remove inConverter from sparql endpoint as that is already the default in the (converting.json)

* fix: add metadatastrategy to RepresentationConvertingStore in regex.json

* test: add integration tests for preserving metadata on PUT

* docs: update release notes and adding documentation about preserving metadata on PUT

* WIP: Template create setRepresentation

* fix: Move container exists and not allowed check to setRepresentation

* test: fix lint

* fix: update configs and documentation

* refactor: update and add documentation + small refactoring

* refactor: update and add documentation + small refactoring + fix tests

* fix: Dynamic pod config + tests

* fix: TemplatedResourcesGenerator does not create containers when they already exist

* fix: metadata preservation now deals with complex content types

* docs: explain the case when there is no content-type

* fix: minor comments
2022-08-04 15:24:36 +02:00
Thomas Dupont
5399e75ae4 feat: regexRoutes stored as ordered array 2022-05-18 09:49:51 +02:00
Wannes Kerckhove
fbbccb0cf1 fix: Stop creating meta files for each new resource #1217 2022-05-06 08:20:55 +02:00
Joachim Van Herwegen
77eeae9c9e chore: Update configs to v5.0.0 2022-05-05 11:20:53 +02:00
Joachim Van Herwegen
cfdd1221f0 fix: Always define @type in configs 2022-04-25 09:09:39 +02:00
Joachim Van Herwegen
ce27bec207 fix: Update prefixes in all configs 2022-04-25 09:09:39 +02:00
Joachim Van Herwegen
784bfc8c21 chore: Update configs to v4.0.0 2022-04-19 15:13:55 +02:00
Joachim Van Herwegen
30ad3015f0 refactor: Move key/value storages to relevant configs 2022-03-28 11:43:14 +02:00
Joachim Van Herwegen
a1a6ce01fa feat: Create separate storage to generate keys 2022-03-28 11:43:14 +02:00
Joachim Van Herwegen
0e4d012086 feat: Add CachedResourceSet
Uses a WeakMap on the identifier to cache resource existence.
2022-03-18 14:15:12 +01:00
Joachim Van Herwegen
b292cd2e2b chore: Update configs to v3.0.0 2022-02-23 11:09:41 +01:00
Thomas Dupont
d067165b68
fix: Split AccountStorage and ForgotPasswordStorage (expiring now) 2022-02-09 13:26:42 +01:00
Joachim Van Herwegen
a9941ebe78 feat: Add support for N3 Patch 2022-01-25 11:30:42 +01:00
Joachim Van Herwegen
1afed65368 feat: Return correct status codes for invalid requests 2022-01-21 17:10:38 +01:00
Arthur Joppart
0cb4d7b161
feat: Add support for quota limits
* feat: implemented SizeReporter and FileSizeReporter

* test: FileSizeReporter tests

* feat: added QuotedDataAccessor

* test: added extra test to check recursiveness of filesizereporter

* feat: added QuotaStrategy interface

* feat: further progress in different files

* feat: wrote doc, tests and improved code

* feat: fixed bugs and code is now runnable and buildable

* feat: finished implementation

* fix: revert accidental chanegs

* fix: fileSizeReported did not count container size

* fix: bug calculating container sizes fixed

* test: FileSizeReporter tests

* test: QuotaDataValidator tests

* test: QuotaError tests

* fix: removed console.log

* doc: added doc to several files

* doc: changed doc for QuotaStrategy to new implementation

* fix: improved content length regex

* feat: improved GlobalQuotaStrategy code

* fix: made FileSizeReported readonly

* feat: added comments to quota-file.json

* fix: changed default tempFilePath variable

* test: included new tempFilePath variable in testing

* chore: created seperate command for start:file:quota to pass tests

* feat: removed all sync fs calls from FileSizeReporter

* feat: minor changes in multple files

* fix: changed function signatures to be in line with others

* feat: optimized quota data validation

* feat: improved FileSizeReporter code

* fix: corrected calculation of containersizes and fixed erroring edgecase

* feat: save content-length as number in metadata

* feat: added comments and changed GlobalQuotaStrategy constructor

* feat: changed file names and added small comment

* test: AtomicFileDataAccessor tests

* test: completed FileSizeReporter tests

* fix: content-length is now saved correctly in RepresentationMetadata

* feat: adapted content length metadata + tests

* fix: removed tempFilePath variable

* fix: reverted .gitignore

* fix: forgot to remove tempFilePath variable from componentsjs config

* test: GlobalQuotaStrategy tests

* feat: replaced DataValidator with Validator

* feat: reworked DataValidator

* feat: added calcultateChunkSize() to SizeReporter

* test: updated FileSizeReporter tests

* fix: tempFile location now relative to rootFilePath

* test: QuotaDataValidator tests

* fix: corrected FileSizeReporter tests

* fix: adapted FileSizeReporter tests

* fix: FileSizeReporter bug on Windows

* fix: regex linting error

* feat: changed Validator class

* feat: added PodQuotaStrategy to enable suota on a per pod basis

* chore: bump context versions

* fix: Capitalized comments in json file

* chore: renamed ValidatorArgs to ValidatorInput

* chore: order all exports

* fix: made TODO comment clearer

* chore: added seperated config files for global and pod based quota + fixed comments

* chore: made minor changes to comments

* feat: added PassthroughDataAccessor

* feat: added PasstroughtDataAccessor + tests

* fix: added invalid header check to ContentLengthParser

* chore: improved mocks

* chore: move quota limit higher up in config

* fix: atomicity issue in AtomicFileDataAccessor

* chore: moved .internal folder to config from FileSizeReporter

* fix: improved algorithm to ignore folders while calculating file size in FileSizeReporter

* fix: changes to support containers in the future

* fix: added error handling to prevent reading of unexistent files

* feat: added generic type to SizeReporter to calculate chunk sizes

* test: use mocked DataAccessor

* chore: added some comments to test and made minor improvement

* fix: fs mock rename

* chore: QuotaStrategy.estimateSize refactor

* chore: move trackAvailableSpace to abstract class QuotaStrategy

* fix: improved test case

* test: quota integration tests

* chore: edited some comments

* chore: change lstat to stat

* feat: moved estimateSize to SizeReporter to be consistent with calcultateChunkSize

* test: finish up tests to reach coverage

* fix: basic config

* fix: minor changes to test CI run

* fix: small fix for windows

* fix: improved writing to file

* chore: linting errors

* chore: rename trackAvailableSpace

* test: improved integration tests

* test: logging info for test debugging

* test: extra logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: improved Quota integration test setup

* test: improve quota tests for CI run

* test: debugging Quota test

* test: uncommented global quota test

* test: changed global quota parameters

* test: logging for debugging

* test: logging cleanup

* chore: minor changes, mostly typo fixes

* chore: remove console.log

* fix: getting inconsistent results

* chore: try fix index.ts CI error

* chore: try fix CI error

* chore: try fix CI error

* chore: revert last commits

* chore: fix inconsistent files with origin

* test: minor test improvements

* chore: minor refactors and improvements

* fix: added extra try catch for breaking bug

* chore: improve config

* chore: minor code improvements

* test: use mockFs

* feat: add extra check in podQuotaStrategy

* chore: replace handle by handleSafe in ValidatingDataAccessor

* chore: typo

* test: improved Quota integration tests

* test: made comment in test more correct

* fix: rm -> rmdir for backwards compatibility

* fix: fsPromises issue

* chore: leave out irrelevant config

* chore: removed start script from package.json

* fix: Small fixes

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2022-01-21 10:49:05 +01:00
Ruben Verborgh
8c9887feea fix: Do not serve UI on special pages.
Fixes https://github.com/solid/community-server/issues/1034
2021-11-02 15:07:22 +01:00
Ruben Verborgh
6d985a65d0 chore: Update configs to v2.0.0. 2021-10-15 10:50:32 +02:00
Joachim Van Herwegen
bf28c83ffa feat: Use PermissionReaders to determine available permissions
These readers will determine which permissions
are available for the incoming credentials.
Their results then get combined in a UnionReader
and authorized in a PermissionBasedAuthorizer
2021-09-28 13:06:38 +02:00
Joachim Van Herwegen
b592d449eb feat: Integrate setup behaviour
This adds options for enabling setup to the config folder.
All default configs with permanent storage (file/sparql)
are configured to require setup at server start.
Memory-based configs merely have it as an option.
2021-09-27 10:32:34 +02:00
Joachim Van Herwegen
ef9703e284 feat: Patch containers by recreating Representation from metadata
Also included is a change to the Patching architecture.
Patching is now done by RepresentationPatchers that take a Representation as input.
2021-09-16 13:39:11 +02:00
Joachim Van Herwegen
0271133d33 fix: Hide internal data by making it auxiliary 2021-09-14 10:51:37 +02:00
Joachim Van Herwegen
1e1edd5c67 refactor: Clean up internal storage
Each IDP class using storage now has a different storage.
This way those classes don't have to worry about clashing keys anymore.

All internal storage is now in the /.internal/ container,
thereby making it easier to take the location of the internal data into account:
only 1 path needs to be blocked and a regex router handling internal data
differently only has to match 1 path as well.
2021-09-08 10:01:51 +02:00
Joachim Van Herwegen
f34e124e1b fix: Convert data to SparqlDataAccessor in regex config 2021-08-30 09:17:50 +02:00
Joachim Van Herwegen
523390e444 fix: Remove passthrough example for index store 2021-07-28 14:01:01 +02:00
Joachim Van Herwegen
7b94b71e7e feat: Prevent access to internal storage containers 2021-07-27 14:31:02 +02:00
Joachim Van Herwegen
45f9a51d7c fix: Prevent cyclical dependency with locker and storage 2021-07-27 13:24:15 +02:00
Ruben Verborgh
6495d650c2 chore: Use consistent strategy naming. 2021-07-21 23:37:23 +02:00
Joachim Van Herwegen
ad7f4ed134 feat: Split ResourceStore config into 2 parts 2021-07-16 16:10:04 +02:00
Joachim Van Herwegen
cc1e332394 feat: Add IndexRepresentationStore to support index resources 2021-07-15 11:21:07 +02:00
Ruben Verborgh
d01382d36e Release version 1.0.0-beta.0 of the npm package. 2021-06-29 17:43:44 +01:00
Joachim Van Herwegen
3a0cf60a55 fix: Support root container access in regex.json 2021-06-25 14:34:31 +02:00
Joachim Van Herwegen
478a915790 fix: Remove source parameters from individual store configs 2021-06-17 10:44:43 +02:00
Joachim Van Herwegen
452032e312 feat: Split preset configurations by feature 2021-05-19 09:40:40 +02:00