225 Commits

Author SHA1 Message Date
Joachim Van Herwegen
e6519992bf Merge branch 'main' into versions/4.0.0
# Conflicts:
#	test/unit/storage/accessors/FileDataAccessor.test.ts
2022-04-15 10:28:50 +02:00
Jasper Vaneessen
668d0a331f
feat: Only accept NamedNodes as predicates for metadata
* refactor: move toCachedNamedNode (private)

* chore: only NamedNodes predicates in removes

* feat: enforce NamedNode predicates in most cases

* feat: getAll only accepts NamedNodes

* feat: toCachedNamedNode only accepts string arg

* tests: use NamedNodes for getAll calls

* test: remove unnecessary string check for coverage

* tests: fix NamedNodes in new tests after rebase

* feat: metadatawriters store NamedNodes

* refactor: toCachedNamedNode as utility function

* fix: double write of linkRelMap

* test: use the CONTENT_TYPE constant
2022-04-15 09:53:39 +02:00
Jasper Vaneessen
fe39f97ee0
refactor: Use fs-extra instead of fs to simplify file access
* refactor: use fs-extra instead of fs

* tests: manual mocks for fs-extra base + ensureDir

* refactor: mockFileSystem + mockFs and mockFsExtra

* add remove mock and some further test tweaks

* test: FileDataAccessor tests passing

* refactor: remove try-catch due to fs-extra handlin

* refactor: fs-extra in atomicFileDataAccessor

* refactor: AtomicFileDataAccessor fs-extra

* test: fix coverage

* refactor: use read/writeJson from fs-extra

* refactor: less duplicate mocking code

* refactor: re-use opendir mocking code
2022-04-12 11:02:30 +02:00
Joachim Van Herwegen
16e9368734 test: Add flushPromises utility function 2022-03-31 17:00:09 +02:00
Joachim Van Herwegen
76548011f2 fix: Support entries function in JsonResourceStorage 2022-03-31 17:00:09 +02:00
Joachim Van Herwegen
641d39d4bc Merge branch 'main' into versions/4.0.0 2022-03-29 14:54:59 +02:00
Joachim Van Herwegen
a1a6ce01fa feat: Create separate storage to generate keys 2022-03-28 11:43:14 +02:00
Joachim Van Herwegen
027e3707fd feat: Parse content-type more strictly 2022-03-28 10:45:05 +02:00
Joachim Van Herwegen
7152897b89 fix: Add IANA type to child metadata in FileDataAccessor 2022-03-28 10:45:05 +02:00
Joachim Van Herwegen
0e4d012086 feat: Add CachedResourceSet
Uses a WeakMap on the identifier to cache resource existence.
2022-03-18 14:15:12 +01:00
Joachim Van Herwegen
4404fa07d9 refactor: Rename resourceExists to hasResource
The function was also moved to the smaller interface ResourceSet.
2022-03-18 14:15:12 +01:00
Joachim Van Herwegen
e4f1e2aa0c chore(deps): update arrayify-stream to v2 2022-03-10 09:01:53 +01:00
Thomas Dupont
5965268ebf
fix: Prevent slugs with trailing slashes for non-container resources
* fix: slugs ending on slash with link header cannot create containers

* refactor(DataAccessorBasedStore): removing duplicate code and errors

* test(ServerFetch): fix integration tests that create a container

* fix: Reinstate original checks in setResource to let tests succeed

* test: change to container paths to end in / on POST only

* refactor: incorporate review changes

* fix: Error check was too strict

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* test: make testcase to test allowed slash behaviour

* test: removed unnecessary code from tests

* test: remove metadata line and duplicate tests

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2022-02-23 14:35:02 +01:00
Ruben Verborgh
6306090114 refactor: Add resolveModulePath. 2022-02-22 09:31:24 +01:00
Joachim Van Herwegen
c5052625d1 Merge branch 'main' into versions/3.0.0
# Conflicts:
#	package-lock.json
#	package.json
#	src/storage/patch/SparqlUpdatePatcher.ts
#	test/unit/init/AppRunner.test.ts
#	test/unit/util/QuadUtil.test.ts
2022-02-11 15:30:45 +01:00
Joachim Van Herwegen
d2870e5c8b fix(deps): Update dependencies 2022-02-11 15:28:50 +01:00
Joachim Van Herwegen
9ceb62b8b6 refactor: Remove @rdfjs/data-model dependency
We already have N3.js for term generation.
No need to have two solutions there.
2022-02-11 15:28:50 +01:00
Joachim Van Herwegen
90a6460c8d Merge branch 'main' into versions/3.0.0
# Conflicts:
#	package-lock.json
#	test/integration/Identity.test.ts
#	test/integration/RepresentationConverter.test.ts
2022-01-25 11:44:24 +01:00
Joachim Van Herwegen
a9941ebe78 feat: Add support for N3 Patch 2022-01-25 11:30:42 +01:00
Joachim Van Herwegen
af049124bf chore: Update eslint dependencies 2022-01-24 09:37:50 +01:00
Arthur Joppart
0cb4d7b161
feat: Add support for quota limits
* feat: implemented SizeReporter and FileSizeReporter

* test: FileSizeReporter tests

* feat: added QuotedDataAccessor

* test: added extra test to check recursiveness of filesizereporter

* feat: added QuotaStrategy interface

* feat: further progress in different files

* feat: wrote doc, tests and improved code

* feat: fixed bugs and code is now runnable and buildable

* feat: finished implementation

* fix: revert accidental chanegs

* fix: fileSizeReported did not count container size

* fix: bug calculating container sizes fixed

* test: FileSizeReporter tests

* test: QuotaDataValidator tests

* test: QuotaError tests

* fix: removed console.log

* doc: added doc to several files

* doc: changed doc for QuotaStrategy to new implementation

* fix: improved content length regex

* feat: improved GlobalQuotaStrategy code

* fix: made FileSizeReported readonly

* feat: added comments to quota-file.json

* fix: changed default tempFilePath variable

* test: included new tempFilePath variable in testing

* chore: created seperate command for start:file:quota to pass tests

* feat: removed all sync fs calls from FileSizeReporter

* feat: minor changes in multple files

* fix: changed function signatures to be in line with others

* feat: optimized quota data validation

* feat: improved FileSizeReporter code

* fix: corrected calculation of containersizes and fixed erroring edgecase

* feat: save content-length as number in metadata

* feat: added comments and changed GlobalQuotaStrategy constructor

* feat: changed file names and added small comment

* test: AtomicFileDataAccessor tests

* test: completed FileSizeReporter tests

* fix: content-length is now saved correctly in RepresentationMetadata

* feat: adapted content length metadata + tests

* fix: removed tempFilePath variable

* fix: reverted .gitignore

* fix: forgot to remove tempFilePath variable from componentsjs config

* test: GlobalQuotaStrategy tests

* feat: replaced DataValidator with Validator

* feat: reworked DataValidator

* feat: added calcultateChunkSize() to SizeReporter

* test: updated FileSizeReporter tests

* fix: tempFile location now relative to rootFilePath

* test: QuotaDataValidator tests

* fix: corrected FileSizeReporter tests

* fix: adapted FileSizeReporter tests

* fix: FileSizeReporter bug on Windows

* fix: regex linting error

* feat: changed Validator class

* feat: added PodQuotaStrategy to enable suota on a per pod basis

* chore: bump context versions

* fix: Capitalized comments in json file

* chore: renamed ValidatorArgs to ValidatorInput

* chore: order all exports

* fix: made TODO comment clearer

* chore: added seperated config files for global and pod based quota + fixed comments

* chore: made minor changes to comments

* feat: added PassthroughDataAccessor

* feat: added PasstroughtDataAccessor + tests

* fix: added invalid header check to ContentLengthParser

* chore: improved mocks

* chore: move quota limit higher up in config

* fix: atomicity issue in AtomicFileDataAccessor

* chore: moved .internal folder to config from FileSizeReporter

* fix: improved algorithm to ignore folders while calculating file size in FileSizeReporter

* fix: changes to support containers in the future

* fix: added error handling to prevent reading of unexistent files

* feat: added generic type to SizeReporter to calculate chunk sizes

* test: use mocked DataAccessor

* chore: added some comments to test and made minor improvement

* fix: fs mock rename

* chore: QuotaStrategy.estimateSize refactor

* chore: move trackAvailableSpace to abstract class QuotaStrategy

* fix: improved test case

* test: quota integration tests

* chore: edited some comments

* chore: change lstat to stat

* feat: moved estimateSize to SizeReporter to be consistent with calcultateChunkSize

* test: finish up tests to reach coverage

* fix: basic config

* fix: minor changes to test CI run

* fix: small fix for windows

* fix: improved writing to file

* chore: linting errors

* chore: rename trackAvailableSpace

* test: improved integration tests

* test: logging info for test debugging

* test: extra logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: improved Quota integration test setup

* test: improve quota tests for CI run

* test: debugging Quota test

* test: uncommented global quota test

* test: changed global quota parameters

* test: logging for debugging

* test: logging cleanup

* chore: minor changes, mostly typo fixes

* chore: remove console.log

* fix: getting inconsistent results

* chore: try fix index.ts CI error

* chore: try fix CI error

* chore: try fix CI error

* chore: revert last commits

* chore: fix inconsistent files with origin

* test: minor test improvements

* chore: minor refactors and improvements

* fix: added extra try catch for breaking bug

* chore: improve config

* chore: minor code improvements

* test: use mockFs

* feat: add extra check in podQuotaStrategy

* chore: replace handle by handleSafe in ValidatingDataAccessor

* chore: typo

* test: improved Quota integration tests

* test: made comment in test more correct

* fix: rm -> rmdir for backwards compatibility

* fix: fsPromises issue

* chore: leave out irrelevant config

* chore: removed start script from package.json

* fix: Small fixes

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2022-01-21 10:49:05 +01:00
Joachim Van Herwegen
11192ed4df Merge branch 'main' into versions/3.0.0 2021-11-22 09:20:41 +01:00
Joachim Van Herwegen
4d319d2564 fix: Handle JSON preferences correctly in dynamic converter 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
c6544fac1d fix: Prefer all inputs equally when generating quads 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
d52aa94e53 feat: Add "no conversion" as possible path in ChainedConverter 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
fdd42bb7b3 feat: Add ContentTypeReplacer to conversion chain 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
fa94c7d4bb feat: Determine Typed Converter output based on input type 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
27306d6e3f refactor: Create BaseTypedRepresentationConverter 2021-11-08 10:16:28 +01:00
Renovate Bot
2e99cc9934 fix(deps): update dependency sparqlalgebrajs to v4 2021-11-05 16:07:20 +01:00
Ruben Verborgh
2e4589938f
feat: Display symlinks in container listings.
Closes https://github.com/solid/community-server/issues/1015
2021-10-21 17:26:54 +01:00
Joachim Van Herwegen
fb3a59c054 fix: Return 409 when there is a slash semantics issue 2021-10-15 09:15:49 +02:00
Joachim Van Herwegen
5613ff9e71 fix: Let Representations always have a body
This is relevant when the request has a content-type
but no data.
2021-10-12 13:30:06 +02:00
Joachim Van Herwegen
b3da9c9fcf refactor: Restructure source code folder
This way the location of certain classes should make more sense
2021-10-12 12:51:02 +02:00
Joachim Van Herwegen
7c7fee5f5c refactor: Rename associated resource to subject resource 2021-10-08 15:49:49 +02:00
Joachim Van Herwegen
7f8b923399 feat: Replace acl specific permissions with generic permissions
This required AuxiliaryStrategy to have a new function
indicating if the auxiliary resource just used its associated resource authorization
or its own.
2021-09-28 13:06:38 +02:00
Joachim Van Herwegen
ef9703e284 feat: Patch containers by recreating Representation from metadata
Also included is a change to the Patching architecture.
Patching is now done by RepresentationPatchers that take a Representation as input.
2021-09-16 13:39:11 +02:00
Joachim Van Herwegen
a1c3633a25 fix: Only check DataAccessor canHandle call for Documents 2021-09-16 13:39:11 +02:00
Joachim Van Herwegen
0271133d33 fix: Hide internal data by making it auxiliary 2021-09-14 10:51:37 +02:00
Joachim Van Herwegen
cc1c3d9223 feat: Support JSON errors
The IDP behaviour has been changed to move all error related knowledge
to the IdentityProviderHttpHandler instead of managing it
in the Interactionhandlers.
2021-09-08 13:55:52 +02:00
Joachim Van Herwegen
4f1a86dfa0 feat: Convert IDP input data to JSON 2021-09-08 13:55:52 +02:00
Joachim Van Herwegen
7f3eab0b20 fix: Prevent parent containers from storing generated metadata 2021-09-08 10:19:28 +02:00
Joachim Van Herwegen
0b4e688359 Merge branch 'main' into versions/2.0.0 2021-09-03 10:09:03 +02:00
Joachim Van Herwegen
ab06dd30f3 feat: Allow filtering in ConstantConverter based on type 2021-09-02 07:56:24 +02:00
Joachim Van Herwegen
ff200e22a9 feat: Add a cache to the AgentGroupAccessChecker 2021-08-23 16:39:25 +02:00
Joachim Van Herwegen
80ebd02cc4 feat: Support content negotiation for IDP requests 2021-08-18 15:39:02 +02:00
Joachim Van Herwegen
0d42987bbd feat: Verify conditions in DataAccessorBasedStore 2021-08-18 13:16:08 +02:00
Joachim Van Herwegen
20f783a581 feat: Create conditions based on input headers 2021-08-18 13:16:08 +02:00
Joachim Van Herwegen
97c534b2bf feat: Keep track of last modified date of resources 2021-08-18 13:16:08 +02:00
Ruben Verborgh
1719857e4b change: Use @css: instead of $PACKAGE_ROOT/
This makes usage from the command line easier,
since $ is typically used to mark variables.
2021-08-03 13:19:38 +01:00
Joachim Van Herwegen
18a71032c0 feat: Create ChainedTemplateEngine for combining engines 2021-08-03 12:52:30 +02:00