884 Commits

Author SHA1 Message Date
Arthur Joppart
0cb4d7b161
feat: Add support for quota limits
* feat: implemented SizeReporter and FileSizeReporter

* test: FileSizeReporter tests

* feat: added QuotedDataAccessor

* test: added extra test to check recursiveness of filesizereporter

* feat: added QuotaStrategy interface

* feat: further progress in different files

* feat: wrote doc, tests and improved code

* feat: fixed bugs and code is now runnable and buildable

* feat: finished implementation

* fix: revert accidental chanegs

* fix: fileSizeReported did not count container size

* fix: bug calculating container sizes fixed

* test: FileSizeReporter tests

* test: QuotaDataValidator tests

* test: QuotaError tests

* fix: removed console.log

* doc: added doc to several files

* doc: changed doc for QuotaStrategy to new implementation

* fix: improved content length regex

* feat: improved GlobalQuotaStrategy code

* fix: made FileSizeReported readonly

* feat: added comments to quota-file.json

* fix: changed default tempFilePath variable

* test: included new tempFilePath variable in testing

* chore: created seperate command for start:file:quota to pass tests

* feat: removed all sync fs calls from FileSizeReporter

* feat: minor changes in multple files

* fix: changed function signatures to be in line with others

* feat: optimized quota data validation

* feat: improved FileSizeReporter code

* fix: corrected calculation of containersizes and fixed erroring edgecase

* feat: save content-length as number in metadata

* feat: added comments and changed GlobalQuotaStrategy constructor

* feat: changed file names and added small comment

* test: AtomicFileDataAccessor tests

* test: completed FileSizeReporter tests

* fix: content-length is now saved correctly in RepresentationMetadata

* feat: adapted content length metadata + tests

* fix: removed tempFilePath variable

* fix: reverted .gitignore

* fix: forgot to remove tempFilePath variable from componentsjs config

* test: GlobalQuotaStrategy tests

* feat: replaced DataValidator with Validator

* feat: reworked DataValidator

* feat: added calcultateChunkSize() to SizeReporter

* test: updated FileSizeReporter tests

* fix: tempFile location now relative to rootFilePath

* test: QuotaDataValidator tests

* fix: corrected FileSizeReporter tests

* fix: adapted FileSizeReporter tests

* fix: FileSizeReporter bug on Windows

* fix: regex linting error

* feat: changed Validator class

* feat: added PodQuotaStrategy to enable suota on a per pod basis

* chore: bump context versions

* fix: Capitalized comments in json file

* chore: renamed ValidatorArgs to ValidatorInput

* chore: order all exports

* fix: made TODO comment clearer

* chore: added seperated config files for global and pod based quota + fixed comments

* chore: made minor changes to comments

* feat: added PassthroughDataAccessor

* feat: added PasstroughtDataAccessor + tests

* fix: added invalid header check to ContentLengthParser

* chore: improved mocks

* chore: move quota limit higher up in config

* fix: atomicity issue in AtomicFileDataAccessor

* chore: moved .internal folder to config from FileSizeReporter

* fix: improved algorithm to ignore folders while calculating file size in FileSizeReporter

* fix: changes to support containers in the future

* fix: added error handling to prevent reading of unexistent files

* feat: added generic type to SizeReporter to calculate chunk sizes

* test: use mocked DataAccessor

* chore: added some comments to test and made minor improvement

* fix: fs mock rename

* chore: QuotaStrategy.estimateSize refactor

* chore: move trackAvailableSpace to abstract class QuotaStrategy

* fix: improved test case

* test: quota integration tests

* chore: edited some comments

* chore: change lstat to stat

* feat: moved estimateSize to SizeReporter to be consistent with calcultateChunkSize

* test: finish up tests to reach coverage

* fix: basic config

* fix: minor changes to test CI run

* fix: small fix for windows

* fix: improved writing to file

* chore: linting errors

* chore: rename trackAvailableSpace

* test: improved integration tests

* test: logging info for test debugging

* test: extra logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: improved Quota integration test setup

* test: improve quota tests for CI run

* test: debugging Quota test

* test: uncommented global quota test

* test: changed global quota parameters

* test: logging for debugging

* test: logging cleanup

* chore: minor changes, mostly typo fixes

* chore: remove console.log

* fix: getting inconsistent results

* chore: try fix index.ts CI error

* chore: try fix CI error

* chore: try fix CI error

* chore: revert last commits

* chore: fix inconsistent files with origin

* test: minor test improvements

* chore: minor refactors and improvements

* fix: added extra try catch for breaking bug

* chore: improve config

* chore: minor code improvements

* test: use mockFs

* feat: add extra check in podQuotaStrategy

* chore: replace handle by handleSafe in ValidatingDataAccessor

* chore: typo

* test: improved Quota integration tests

* test: made comment in test more correct

* fix: rm -> rmdir for backwards compatibility

* fix: fsPromises issue

* chore: leave out irrelevant config

* chore: removed start script from package.json

* fix: Small fixes

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2022-01-21 10:49:05 +01:00
Lina
9a1f324685
feat: Create VoidLocker to disable locking resources
* add: Add VoidLocker and unittest

* Update src/util/locking/VoidLocker.ts

Co-authored-by: Ruben Verborgh <ruben@verborgh.org>

* Update src/util/locking/VoidLocker.ts

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* update: noop function and add debug void config

* add: debug-void in readme

* Update RELEASE_NOTES.md

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* Update config/util/README.md

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* add: missing line

Co-authored-by: lina <lina7906@gmail.com>
Co-authored-by: Ruben Verborgh <ruben@verborgh.org>
Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2021-12-16 15:09:58 +01:00
Joachim Van Herwegen
4241c5348d feat: Move redirect support from IDP handler to specific handlers 2021-12-09 14:10:42 +01:00
Joachim Van Herwegen
7163a0317b feat: Support redirection through errors 2021-12-09 14:10:42 +01:00
Joachim Van Herwegen
520e4fe42f feat: Move OIDC library behaviour to separate path 2021-12-09 13:15:05 +01:00
Joachim Van Herwegen
11192ed4df Merge branch 'main' into versions/3.0.0 2021-11-22 09:20:41 +01:00
Ruben Verborgh
5a01f09f81 refactor: Simplify promiseSome. 2021-11-18 08:41:37 +00:00
Ruben Verborgh
c13456c225 fix: Make UnionCredentialsExtractor tolerate failures.
Fixes https://github.com/solid/community-server/issues/1031
2021-11-11 12:48:39 +00:00
Joachim Van Herwegen
a1579f625a chore: Create pull request template 2021-11-08 13:59:29 +01:00
Joachim Van Herwegen
fc60b5c161 feat: Update IDP parameters to latest Solid-OIDC version 2021-11-08 11:39:16 +01:00
Joachim Van Herwegen
b0208bf469 docs: Update release notes with conversion changes 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
4d319d2564 fix: Handle JSON preferences correctly in dynamic converter 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
c6544fac1d fix: Prefer all inputs equally when generating quads 2021-11-08 10:16:28 +01:00
Ruben Verborgh
ed287ffade test: Add content negotiation integration tests 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
d52aa94e53 feat: Add "no conversion" as possible path in ChainedConverter 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
fdd42bb7b3 feat: Add ContentTypeReplacer to conversion chain 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
fa94c7d4bb feat: Determine Typed Converter output based on input type 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
27306d6e3f refactor: Create BaseTypedRepresentationConverter 2021-11-08 10:16:28 +01:00
Joachim Van Herwegen
06663f3079 chore: Remove docker references from Windows integration tests 2021-11-05 16:19:54 +01:00
Joachim Van Herwegen
c2d811754b chore: Run integration tests on Windows
Also removed 17.0 from unit tests due to being a non-LTS version
2021-11-05 16:13:12 +01:00
Renovate Bot
2e99cc9934 fix(deps): update dependency sparqlalgebrajs to v4 2021-11-05 16:07:20 +01:00
Joachim Van Herwegen
9f0973b039 fix: Linter warnings 2021-11-05 15:59:28 +01:00
Renovate Bot
d317215e08 fix(deps): update dependency ws and @types/ws to v8.2 2021-11-05 14:48:35 +01:00
Joachim Van Herwegen
e9d2b17938 chore: Create issue templates 2021-11-05 09:48:55 +01:00
Joachim Van Herwegen
1810a97f20 chore: Target 3.0.0 branch in conformance CI 2021-11-05 08:50:17 +01:00
Ruben Verborgh
3861e64398 chore: Fix CHANGELOG for v2.0.1. 2021-11-02 16:37:01 +01:00
Ruben Verborgh
c24fe9fba8 Release version 2.0.1 of the npm package. v2.0.1 2021-11-02 15:34:33 +01:00
Ruben Verborgh
8c9887feea fix: Do not serve UI on special pages.
Fixes https://github.com/solid/community-server/issues/1034
2021-11-02 15:07:22 +01:00
Ruben Verborgh
e1a4ef29b9 chore: Run CI on version branches. 2021-11-02 13:36:18 +01:00
Ruben Verborgh
a17731ed1e chore: Run CI on Node 17. 2021-11-02 13:23:40 +01:00
Ruben Verborgh
557ba32577 chore: Gather coverage from all OSes. 2021-11-02 13:23:23 +01:00
Ruben Verborgh
b8db72dfd3 chore: Run integration tests separately. 2021-11-02 12:55:09 +01:00
Ruben Verborgh
81e7694845 chore: Fix dependency vulnerabilities. 2021-11-01 15:17:56 +01:00
Ruben Verborgh
b42150cf52 fix: Allow URLs with multiple leading slashes.
Fixes https://github.com/solid/community-server/issues/1025
2021-10-25 23:42:16 +02:00
Ruben Verborgh
2e4589938f
feat: Display symlinks in container listings.
Closes https://github.com/solid/community-server/issues/1015
2021-10-21 17:26:54 +01:00
Ruben Verborgh
9f241631f8 chore: Ignore test/tmp. 2021-10-21 13:57:28 +01:00
Ruben Verborgh
93b71eb726 docs: Add more warning to root ACLs. 2021-10-18 14:59:42 +01:00
Joachim Van Herwegen
d81b9ef2a4 chore: Add v2.1.0 branch to conformance test run 2021-10-18 10:22:30 +02:00
Wout Slabbinck
54ccbed48d fix: Added a content-type parser to HeaderUtil
* fixes #995

* fixed the remarks and added a test for parseContentType

* refactor: remove unnecessary comment

Co-authored-by: woutslabbinck <wout.slabbinck@ugent.com>
2021-10-15 13:13:37 +02:00
Joachim Van Herwegen
2a48b1f333 chore: Remove v2.0.0 branch from conformance test CI 2021-10-15 11:41:19 +02:00
Joachim Van Herwegen
6f858be350 chore: Update @solid/access-token-verifier to v1.0.1 2021-10-15 11:37:22 +02:00
Ruben Verborgh
216788e225 Release version 2.0.0 of the npm package. v2.0.0 2021-10-15 10:50:44 +02:00
Ruben Verborgh
6d985a65d0 chore: Update configs to v2.0.0. 2021-10-15 10:50:32 +02:00
Joachim Van Herwegen
4da1b1cc7d docs: Document new features and config changes for 2.0.0 2021-10-15 10:30:56 +02:00
Ruben Verborgh
d2fbcdd4d2 docs: Clarify solutions to wrong base URL. 2021-10-15 10:28:42 +02:00
Ruben Verborgh
a188bf1c6f docs: Explain how to log in.
Closes https://github.com/solid/community-server/pull/970
2021-10-15 10:28:32 +02:00
Joachim Van Herwegen
fb3a59c054 fix: Return 409 when there is a slash semantics issue 2021-10-15 09:15:49 +02:00
Joachim Van Herwegen
76c87bb56a fix: Return 201 when creating new resources 2021-10-12 15:44:31 +02:00
Joachim Van Herwegen
5613ff9e71 fix: Let Representations always have a body
This is relevant when the request has a content-type
but no data.
2021-10-12 13:30:06 +02:00
Joachim Van Herwegen
f1ef2ced03 docs: Add routing config for sparql endpoint and storage
Due to key/value stores using JSON objects,
the SPARQL resource store can not be used to store them,
meaning the only out of the box solution that works is memory storage.
This config makes it so file storage can be used for internal storage
while still having a SPARQL backend.
2021-10-12 13:13:09 +02:00