172 Commits

Author SHA1 Message Date
Joachim Van Herwegen
f3dedf4e27 feat: Add utility functions for generating error classes 2022-03-29 14:52:36 +02:00
Ruben Verborgh
e4688e12a1 chore: Simplify test mocks. 2022-03-29 14:41:31 +02:00
Arthur Joppart
4a37f8c879 chore: replace redis library with ioredis 2022-03-29 14:41:31 +02:00
Joachim Van Herwegen
027e3707fd feat: Parse content-type more strictly 2022-03-28 10:45:05 +02:00
Thomas Dupont
a8602055e6
feat: Store content type parameters
* feat: support storage and retrievel of content-type parameters

* test: extra unit tests for parseContentTypeWithParameters

* refactor: simplify set contentType()

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* refactor: simplify for loop because of unique blankNodes

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* refactor: ContentTypeParameter should be contentTypeParameter

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* refactor: remove undefined type in favor of var? syntax

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* refactor: use new parseContentType internally

* chore: remove commented code

* docs: code documentation line changed

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* refactor: Check for faulty metadata in contentType rdf structure

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* refactor: remove all instances of blanknodes

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* refactor: use full contentType when parsing header

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* refactor: use quads() method instead of store.getQuads()

* refactor: .value needed for type correctness

* feat: ReprMetadata constructor now supports full content-type string

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2022-03-14 10:27:34 +01:00
Ruben Verborgh
6306090114 refactor: Add resolveModulePath. 2022-02-22 09:31:24 +01:00
Thomas Dupont
ce754c119f
fix: Add content-negotiation when fetching dataset from url
* Solution works but tests don't

* refactor(FetchUtil): use arrayifyStream

* refactor(FetchUtil): split fetchDataset into 2 separate functions

* style(FetchUtil): onelining instead of declaring new local var

* test: trying to mock rdfDereferencer

* refactor: promise can't have async function as arg

* test(FetchUtil): pass Quad array to mockDereference instead

* test: all tests should pass now and coverage is back to 100%

* style: comment typo

* chore: make package.json and package-lock.json compatible with main

* chore: fix package.json double entries

* chore: updated package.json to be alfabetical again

* refactor(AgentGroupAccessChecker): Remove converter from contructor and config

* refactor(TokenOwnerShipValidator): Remove converter from constructor and config

* refactor(FetchUtil): Return BadRequestHttpError instead of generic Error

* test(FetchUtil): return Response object instead of mocking fetch

* style: typos and newlines
2022-02-15 13:44:03 +01:00
Joachim Van Herwegen
c5052625d1 Merge branch 'main' into versions/3.0.0
# Conflicts:
#	package-lock.json
#	package.json
#	src/storage/patch/SparqlUpdatePatcher.ts
#	test/unit/init/AppRunner.test.ts
#	test/unit/util/QuadUtil.test.ts
2022-02-11 15:30:45 +01:00
Joachim Van Herwegen
d2870e5c8b fix(deps): Update dependencies 2022-02-11 15:28:50 +01:00
Joachim Van Herwegen
9ceb62b8b6 refactor: Remove @rdfjs/data-model dependency
We already have N3.js for term generation.
No need to have two solutions there.
2022-02-11 15:28:50 +01:00
Joachim Van Herwegen
90a6460c8d Merge branch 'main' into versions/3.0.0
# Conflicts:
#	package-lock.json
#	test/integration/Identity.test.ts
#	test/integration/RepresentationConverter.test.ts
2022-01-25 11:44:24 +01:00
Joachim Van Herwegen
a9941ebe78 feat: Add support for N3 Patch 2022-01-25 11:30:42 +01:00
Joachim Van Herwegen
af049124bf chore: Update eslint dependencies 2022-01-24 09:37:50 +01:00
Joachim Van Herwegen
1afed65368 feat: Return correct status codes for invalid requests 2022-01-21 17:10:38 +01:00
Arthur Joppart
0cb4d7b161
feat: Add support for quota limits
* feat: implemented SizeReporter and FileSizeReporter

* test: FileSizeReporter tests

* feat: added QuotedDataAccessor

* test: added extra test to check recursiveness of filesizereporter

* feat: added QuotaStrategy interface

* feat: further progress in different files

* feat: wrote doc, tests and improved code

* feat: fixed bugs and code is now runnable and buildable

* feat: finished implementation

* fix: revert accidental chanegs

* fix: fileSizeReported did not count container size

* fix: bug calculating container sizes fixed

* test: FileSizeReporter tests

* test: QuotaDataValidator tests

* test: QuotaError tests

* fix: removed console.log

* doc: added doc to several files

* doc: changed doc for QuotaStrategy to new implementation

* fix: improved content length regex

* feat: improved GlobalQuotaStrategy code

* fix: made FileSizeReported readonly

* feat: added comments to quota-file.json

* fix: changed default tempFilePath variable

* test: included new tempFilePath variable in testing

* chore: created seperate command for start:file:quota to pass tests

* feat: removed all sync fs calls from FileSizeReporter

* feat: minor changes in multple files

* fix: changed function signatures to be in line with others

* feat: optimized quota data validation

* feat: improved FileSizeReporter code

* fix: corrected calculation of containersizes and fixed erroring edgecase

* feat: save content-length as number in metadata

* feat: added comments and changed GlobalQuotaStrategy constructor

* feat: changed file names and added small comment

* test: AtomicFileDataAccessor tests

* test: completed FileSizeReporter tests

* fix: content-length is now saved correctly in RepresentationMetadata

* feat: adapted content length metadata + tests

* fix: removed tempFilePath variable

* fix: reverted .gitignore

* fix: forgot to remove tempFilePath variable from componentsjs config

* test: GlobalQuotaStrategy tests

* feat: replaced DataValidator with Validator

* feat: reworked DataValidator

* feat: added calcultateChunkSize() to SizeReporter

* test: updated FileSizeReporter tests

* fix: tempFile location now relative to rootFilePath

* test: QuotaDataValidator tests

* fix: corrected FileSizeReporter tests

* fix: adapted FileSizeReporter tests

* fix: FileSizeReporter bug on Windows

* fix: regex linting error

* feat: changed Validator class

* feat: added PodQuotaStrategy to enable suota on a per pod basis

* chore: bump context versions

* fix: Capitalized comments in json file

* chore: renamed ValidatorArgs to ValidatorInput

* chore: order all exports

* fix: made TODO comment clearer

* chore: added seperated config files for global and pod based quota + fixed comments

* chore: made minor changes to comments

* feat: added PassthroughDataAccessor

* feat: added PasstroughtDataAccessor + tests

* fix: added invalid header check to ContentLengthParser

* chore: improved mocks

* chore: move quota limit higher up in config

* fix: atomicity issue in AtomicFileDataAccessor

* chore: moved .internal folder to config from FileSizeReporter

* fix: improved algorithm to ignore folders while calculating file size in FileSizeReporter

* fix: changes to support containers in the future

* fix: added error handling to prevent reading of unexistent files

* feat: added generic type to SizeReporter to calculate chunk sizes

* test: use mocked DataAccessor

* chore: added some comments to test and made minor improvement

* fix: fs mock rename

* chore: QuotaStrategy.estimateSize refactor

* chore: move trackAvailableSpace to abstract class QuotaStrategy

* fix: improved test case

* test: quota integration tests

* chore: edited some comments

* chore: change lstat to stat

* feat: moved estimateSize to SizeReporter to be consistent with calcultateChunkSize

* test: finish up tests to reach coverage

* fix: basic config

* fix: minor changes to test CI run

* fix: small fix for windows

* fix: improved writing to file

* chore: linting errors

* chore: rename trackAvailableSpace

* test: improved integration tests

* test: logging info for test debugging

* test: extra logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: improved Quota integration test setup

* test: improve quota tests for CI run

* test: debugging Quota test

* test: uncommented global quota test

* test: changed global quota parameters

* test: logging for debugging

* test: logging cleanup

* chore: minor changes, mostly typo fixes

* chore: remove console.log

* fix: getting inconsistent results

* chore: try fix index.ts CI error

* chore: try fix CI error

* chore: try fix CI error

* chore: revert last commits

* chore: fix inconsistent files with origin

* test: minor test improvements

* chore: minor refactors and improvements

* fix: added extra try catch for breaking bug

* chore: improve config

* chore: minor code improvements

* test: use mockFs

* feat: add extra check in podQuotaStrategy

* chore: replace handle by handleSafe in ValidatingDataAccessor

* chore: typo

* test: improved Quota integration tests

* test: made comment in test more correct

* fix: rm -> rmdir for backwards compatibility

* fix: fsPromises issue

* chore: leave out irrelevant config

* chore: removed start script from package.json

* fix: Small fixes

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2022-01-21 10:49:05 +01:00
Lina
9a1f324685
feat: Create VoidLocker to disable locking resources
* add: Add VoidLocker and unittest

* Update src/util/locking/VoidLocker.ts

Co-authored-by: Ruben Verborgh <ruben@verborgh.org>

* Update src/util/locking/VoidLocker.ts

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* update: noop function and add debug void config

* add: debug-void in readme

* Update RELEASE_NOTES.md

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* Update config/util/README.md

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* add: missing line

Co-authored-by: lina <lina7906@gmail.com>
Co-authored-by: Ruben Verborgh <ruben@verborgh.org>
Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2021-12-16 15:09:58 +01:00
Joachim Van Herwegen
7163a0317b feat: Support redirection through errors 2021-12-09 14:10:42 +01:00
Ruben Verborgh
c13456c225 fix: Make UnionCredentialsExtractor tolerate failures.
Fixes https://github.com/solid/community-server/issues/1031
2021-11-11 12:48:39 +00:00
Wout Slabbinck
54ccbed48d fix: Added a content-type parser to HeaderUtil
* fixes #995

* fixed the remarks and added a test for parseContentType

* refactor: remove unnecessary comment

Co-authored-by: woutslabbinck <wout.slabbinck@ugent.com>
2021-10-15 13:13:37 +02:00
Joachim Van Herwegen
b3da9c9fcf refactor: Restructure source code folder
This way the location of certain classes should make more sense
2021-10-12 12:51:02 +02:00
Joachim Van Herwegen
62f026f2bc feat: Create UnionHandler to combine AsyncHandler results 2021-09-28 13:06:38 +02:00
Joachim Van Herwegen
b592d449eb feat: Integrate setup behaviour
This adds options for enabling setup to the config folder.
All default configs with permanent storage (file/sparql)
are configured to require setup at server start.
Memory-based configs merely have it as an option.
2021-09-27 10:32:34 +02:00
Joachim Van Herwegen
facf691e86 feat: Create ConditionalHandler
This handler rejects all inputs once a certain condition is met
2021-09-27 10:32:34 +02:00
Joachim Van Herwegen
fb0b50c997 fix: Make the RouterHandler more robust
It now extracts paths based on the base URL
and allows catching all methods.
2021-09-27 10:32:34 +02:00
Joachim Van Herwegen
42d3ab0a4c fix: Add filename parameter for EJS templates
This is required if we want to include partial templates
2021-09-27 10:32:34 +02:00
Joachim Van Herwegen
cc1c3d9223 feat: Support JSON errors
The IDP behaviour has been changed to move all error related knowledge
to the IdentityProviderHttpHandler instead of managing it
in the Interactionhandlers.
2021-09-08 13:55:52 +02:00
Joachim Van Herwegen
4f1a86dfa0 feat: Convert IDP input data to JSON 2021-09-08 13:55:52 +02:00
Simone Persiani
73867f0827 feat: Add BooleanHandler
Co-Authored-By: Ludovico Granata <Ludogranata@gmail.com>
2021-08-23 16:39:25 +02:00
Simone Persiani
0355673a0f feat: Add function promiseSome
Co-Authored-By: Ludovico Granata <Ludogranata@gmail.com>
2021-08-23 16:39:25 +02:00
Simone Persiani
c13c03ef54 feat: Add function readableToQuads
Co-Authored-By: Ludovico Granata <Ludogranata@gmail.com>
2021-08-23 16:39:25 +02:00
Joachim Van Herwegen
80ebd02cc4 feat: Support content negotiation for IDP requests 2021-08-18 15:39:02 +02:00
Joachim Van Herwegen
0d42987bbd feat: Verify conditions in DataAccessorBasedStore 2021-08-18 13:16:08 +02:00
Joachim Van Herwegen
97c534b2bf feat: Keep track of last modified date of resources 2021-08-18 13:16:08 +02:00
Ruben Verborgh
1719857e4b change: Use @css: instead of $PACKAGE_ROOT/
This makes usage from the command line easier,
since $ is typically used to mark variables.
2021-08-03 13:19:38 +01:00
Joachim Van Herwegen
18a71032c0 feat: Create ChainedTemplateEngine for combining engines 2021-08-03 12:52:30 +02:00
Joachim Van Herwegen
2ae95bd167 refactor: Remove streamify array dependency 2021-08-02 14:28:43 +02:00
Joachim Van Herwegen
63e88578c3 feat: Rewrite TokenOwnershipValidator behaviour to remove @rdfjs/fetch dependency 2021-08-02 12:02:10 +02:00
Joachim Van Herwegen
5f8ec5dd15 fix: Prevent warnings for expected errors 2021-07-29 16:57:50 +02:00
Ruben Verborgh
9628fe98b8 refactor: Align EJS engine with Handlebars. 2021-07-22 11:12:21 +02:00
Ruben Verborgh
19624dc729 refactor: Allow default template in template engines. 2021-07-22 11:12:21 +02:00
Ruben Verborgh
31dcfbf51e chore: Move FetchUtil to util folder. 2021-07-20 15:33:28 +02:00
Ruben Verborgh
b9295f00c2 feat: Default error code to HTTP status code. 2021-07-15 10:22:28 +01:00
Ruben Verborgh
649f7a9a78 refactor: Assign properties directly to error. 2021-07-13 13:25:30 +01:00
Ruben Verborgh
8a62938c18 test: Use individual error assertions. 2021-07-13 13:25:30 +01:00
Joachim Van Herwegen
f2f967ff8a feat: Convert errorCodes using markdown 2021-07-06 16:42:06 +02:00
Joachim Van Herwegen
35a7cf988c feat: Support metadata in multiple graphs 2021-07-05 14:33:20 +02:00
Joachim Van Herwegen
7cfb87e516 feat: Add better support for non-native errors 2021-06-14 15:29:47 +02:00
Joachim Van Herwegen
e1f95877da feat: Create ErrorHandler to convert errors to Representations 2021-06-07 13:54:38 +02:00
renovate[bot]
afc662ca9a
chore: Update (ts-)jest and use modern fake timers where possible
* chore(deps): update dependency jest to v27

* chore: Update (ts-)jest and use modern fake timers where possible

Co-authored-by: Renovate Bot <bot@renovateapp.com>
Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2021-05-31 15:29:14 +02:00
Joachim Van Herwegen
96a07e4853 refactor: Simplify resource metadata generation 2021-05-12 15:27:51 +02:00