36 Commits

Author SHA1 Message Date
Joachim Van Herwegen
3d6e3d2e39 chore: Drop support for Node 12 2022-04-21 14:54:12 +02:00
Thomas Dupont
e2e2d0864f feat: add a Redis based Read-Write Locker
refactor: more elegant way of providing default attemptSettings to constructor

style(jsdoc): rewording of jsdoc comment

fix: RegExp(/regex/) => /regex/

fix: Replace Error with InternalServerError

docs: jsdoc for RedisReadWriteLocker class

feat: make RedisReadWriteLocker a ResourceLocker too

test: coverage back to 100%

refactor: linting fix

style(jsdoc): Add explanation to tryRedisFn() method

refactor: remove RedisResourceLocker

fix: bug in lua script

chore(deps): update ioredis, remove redlock

refactor: removed RedisResourceLocker in favor of generic RedisLocker class

test: add redis lua scripts tests and integrate all 3 redis integration tests in 1

refactor: remove .vscode folder from index

refactor: Add some typing and  change redis references to Redis in comments

refactor: more changes after PR review

refactor: remove redis.json

refactor: rename redis-rw.json to redis.json

docs: added readme and release notes
2022-04-19 13:52:31 +02:00
Joachim Van Herwegen
e6519992bf Merge branch 'main' into versions/4.0.0
# Conflicts:
#	test/unit/storage/accessors/FileDataAccessor.test.ts
2022-04-15 10:28:50 +02:00
Joachim Van Herwegen
db906ae872 test: Add pod seeding integration test 2022-04-14 10:34:04 +02:00
Jasper Vaneessen
fe39f97ee0
refactor: Use fs-extra instead of fs to simplify file access
* refactor: use fs-extra instead of fs

* tests: manual mocks for fs-extra base + ensureDir

* refactor: mockFileSystem + mockFs and mockFsExtra

* add remove mock and some further test tweaks

* test: FileDataAccessor tests passing

* refactor: remove try-catch due to fs-extra handlin

* refactor: fs-extra in atomicFileDataAccessor

* refactor: AtomicFileDataAccessor fs-extra

* test: fix coverage

* refactor: use read/writeJson from fs-extra

* refactor: less duplicate mocking code

* refactor: re-use opendir mocking code
2022-04-12 11:02:30 +02:00
Joachim Van Herwegen
ce5a98b16f test: Add integration test that simulates running 2 hours 2022-03-31 17:00:09 +02:00
Joachim Van Herwegen
16e9368734 test: Add flushPromises utility function 2022-03-31 17:00:09 +02:00
Joachim Van Herwegen
6f83ac5ead test: Create permission table to automate tests 2022-03-18 14:15:12 +01:00
Joachim Van Herwegen
90a6460c8d Merge branch 'main' into versions/3.0.0
# Conflicts:
#	package-lock.json
#	test/integration/Identity.test.ts
#	test/integration/RepresentationConverter.test.ts
2022-01-25 11:44:24 +01:00
Joachim Van Herwegen
a9941ebe78 feat: Add support for N3 Patch 2022-01-25 11:30:42 +01:00
Joachim Van Herwegen
af049124bf chore: Update eslint dependencies 2022-01-24 09:37:50 +01:00
Arthur Joppart
0cb4d7b161
feat: Add support for quota limits
* feat: implemented SizeReporter and FileSizeReporter

* test: FileSizeReporter tests

* feat: added QuotedDataAccessor

* test: added extra test to check recursiveness of filesizereporter

* feat: added QuotaStrategy interface

* feat: further progress in different files

* feat: wrote doc, tests and improved code

* feat: fixed bugs and code is now runnable and buildable

* feat: finished implementation

* fix: revert accidental chanegs

* fix: fileSizeReported did not count container size

* fix: bug calculating container sizes fixed

* test: FileSizeReporter tests

* test: QuotaDataValidator tests

* test: QuotaError tests

* fix: removed console.log

* doc: added doc to several files

* doc: changed doc for QuotaStrategy to new implementation

* fix: improved content length regex

* feat: improved GlobalQuotaStrategy code

* fix: made FileSizeReported readonly

* feat: added comments to quota-file.json

* fix: changed default tempFilePath variable

* test: included new tempFilePath variable in testing

* chore: created seperate command for start:file:quota to pass tests

* feat: removed all sync fs calls from FileSizeReporter

* feat: minor changes in multple files

* fix: changed function signatures to be in line with others

* feat: optimized quota data validation

* feat: improved FileSizeReporter code

* fix: corrected calculation of containersizes and fixed erroring edgecase

* feat: save content-length as number in metadata

* feat: added comments and changed GlobalQuotaStrategy constructor

* feat: changed file names and added small comment

* test: AtomicFileDataAccessor tests

* test: completed FileSizeReporter tests

* fix: content-length is now saved correctly in RepresentationMetadata

* feat: adapted content length metadata + tests

* fix: removed tempFilePath variable

* fix: reverted .gitignore

* fix: forgot to remove tempFilePath variable from componentsjs config

* test: GlobalQuotaStrategy tests

* feat: replaced DataValidator with Validator

* feat: reworked DataValidator

* feat: added calcultateChunkSize() to SizeReporter

* test: updated FileSizeReporter tests

* fix: tempFile location now relative to rootFilePath

* test: QuotaDataValidator tests

* fix: corrected FileSizeReporter tests

* fix: adapted FileSizeReporter tests

* fix: FileSizeReporter bug on Windows

* fix: regex linting error

* feat: changed Validator class

* feat: added PodQuotaStrategy to enable suota on a per pod basis

* chore: bump context versions

* fix: Capitalized comments in json file

* chore: renamed ValidatorArgs to ValidatorInput

* chore: order all exports

* fix: made TODO comment clearer

* chore: added seperated config files for global and pod based quota + fixed comments

* chore: made minor changes to comments

* feat: added PassthroughDataAccessor

* feat: added PasstroughtDataAccessor + tests

* fix: added invalid header check to ContentLengthParser

* chore: improved mocks

* chore: move quota limit higher up in config

* fix: atomicity issue in AtomicFileDataAccessor

* chore: moved .internal folder to config from FileSizeReporter

* fix: improved algorithm to ignore folders while calculating file size in FileSizeReporter

* fix: changes to support containers in the future

* fix: added error handling to prevent reading of unexistent files

* feat: added generic type to SizeReporter to calculate chunk sizes

* test: use mocked DataAccessor

* chore: added some comments to test and made minor improvement

* fix: fs mock rename

* chore: QuotaStrategy.estimateSize refactor

* chore: move trackAvailableSpace to abstract class QuotaStrategy

* fix: improved test case

* test: quota integration tests

* chore: edited some comments

* chore: change lstat to stat

* feat: moved estimateSize to SizeReporter to be consistent with calcultateChunkSize

* test: finish up tests to reach coverage

* fix: basic config

* fix: minor changes to test CI run

* fix: small fix for windows

* fix: improved writing to file

* chore: linting errors

* chore: rename trackAvailableSpace

* test: improved integration tests

* test: logging info for test debugging

* test: extra logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: logging for debugging

* test: improved Quota integration test setup

* test: improve quota tests for CI run

* test: debugging Quota test

* test: uncommented global quota test

* test: changed global quota parameters

* test: logging for debugging

* test: logging cleanup

* chore: minor changes, mostly typo fixes

* chore: remove console.log

* fix: getting inconsistent results

* chore: try fix index.ts CI error

* chore: try fix CI error

* chore: try fix CI error

* chore: revert last commits

* chore: fix inconsistent files with origin

* test: minor test improvements

* chore: minor refactors and improvements

* fix: added extra try catch for breaking bug

* chore: improve config

* chore: minor code improvements

* test: use mockFs

* feat: add extra check in podQuotaStrategy

* chore: replace handle by handleSafe in ValidatingDataAccessor

* chore: typo

* test: improved Quota integration tests

* test: made comment in test more correct

* fix: rm -> rmdir for backwards compatibility

* fix: fsPromises issue

* chore: leave out irrelevant config

* chore: removed start script from package.json

* fix: Small fixes

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2022-01-21 10:49:05 +01:00
Ruben Verborgh
ed287ffade test: Add content negotiation integration tests 2021-11-08 10:16:28 +01:00
Ruben Verborgh
2e4589938f
feat: Display symlinks in container listings.
Closes https://github.com/solid/community-server/issues/1015
2021-10-21 17:26:54 +01:00
Joachim Van Herwegen
13c49045d4 feat: Support acl authorization for IDP components
Configuration has been updated so the IDP requests
also pass through an Authorization component.
A new config option was added to choose
which authorization scheme to use for the IDP.
2021-10-07 15:51:11 +02:00
Joachim Van Herwegen
b592d449eb feat: Integrate setup behaviour
This adds options for enabling setup to the config folder.
All default configs with permanent storage (file/sparql)
are configured to require setup at server start.
Memory-based configs merely have it as an option.
2021-09-27 10:32:34 +02:00
Joachim Van Herwegen
a7a22bf43a test: Create integration tests for conditions 2021-08-18 13:16:08 +02:00
Joachim Van Herwegen
2ae95bd167 refactor: Remove streamify array dependency 2021-08-02 14:28:43 +02:00
Joachim Van Herwegen
5edbbc1958 chore: Update dependencies 2021-06-30 10:24:13 +02:00
Joachim Van Herwegen
7faad0aef0 feat: Support creation of HTTPS server 2021-06-01 14:56:10 +02:00
Joachim Van Herwegen
cae9d54fac feat: Add getChildren function to DataAccessor interface
DataAccessors are now no longer responsible for generating ldp:contains triples.
2021-05-12 15:27:51 +02:00
jaxoncreed
1d65143e89
feat: Add identity provider (#455)
* Add identity provider handler as a dependency

* Temp Identity

* Figured out how to get koa to work

* Hooked up idp to networking

* Feat/idp architecture refactor (#430)

* Logs in with solid oidc

* Refactored Provider

* Attempt to hook up dependencies

* Partial wiring of oidc provider components

* IdP networking now works with architecture

* Interaction Handlers Set Up

* fix: Rename & adapt to CSS

* Included Login Interaction

* Refactored architecture to bind Interaction Policy to HttpHandlers

Co-authored-by: Matthieu Bosquet <matthieubosquet@gmail.com>

* fix: Rebase on master

* fix: DI after rebase

* Reimplemented Routing

* Renamed modules and removed ProviderFactory (#450)

* refactor: Solid IdP DI

* refactor: IdP interaction handler DI

* refactor: IdP interaction waterfall

* refactor: Remove unnecessary legacy URL parse

* fix: Add legacy parse back in

* feat: adapter & fix: handlers

* Removed adapter factory

* fix: refactor IdP

* fix: refactor IdP

* fix: refactor IdP

* feat: Add IdP to file storage config

* fix: Unintended commit

* fix: Components ignore

* feat: Basic resource store adapter

* Partially complete idp routing

* Set up initial routing injection graph

* Clean up ResourceStorageAdapter

* Refactored configuration architecture

* Hooked up Login UIs (#518)

* feat: Use template path & run fileserver

* feat: Use util function to read resource

* Fixed DI JSON-LD context

* fixed rendering

* WebId validator

* Set up persistent storage for loing and register

* Fixed ejs template routing

* Refactored StorageAdapters

* NSS login successful

* Forgot password infrastructure

* Can send email (#557)

* Can send email

* fix: IdP crashes if interaction ID doesn't exist (#587)

* feat: Require an issuer registration token

* fix: Issuer registration token typo in error

* fix: Remove dummy IdP storage adapter

* fix: Remove unused library lodash

* fix: Remove unused library lru-cache

* Production ready keystore

* Ruben comments before clownface removal

* Removed clownface

* Change key value store

* Completed Ruben's comments

* Added comments to each class

* Fixed errors on login

* Ruben feedback

* Refactored out getPostRenderHandler

* Identity provider tests (#622)

* corrected tests lacking <void> on promises

* Added files for all idp tests

* Added unfinished tests for all added files

* ErrorHandlingWaterfallHandler

* RenderEjsHandler and RouterHandler tests

* GetPostRouterHandler and BasicOnErrorHandler tests

* Corrected tests for updates to Idp

* fix: missing export

* fix: audience claim

* Client Id Support (#630)

* Added client_id for the auth challenge

* Update src/identity/storage/ClientWebIdFetchingStorageAdapterFactory.ts

Co-authored-by: Matthieu Bosquet <matthieubosquet@gmail.com>

Co-authored-by: Matthieu Bosquet <matthieubosquet@gmail.com>

* fix: Rebase fixes

* Several minor Idp changes/refactors (#656)

* fix: Minor changes

* refactor: Split EmailPasswordInteractionPolicy

* refactor: Remove ErrorHandlingWaterfallHandler

* refactor: Clean up dependencies

* fix: Add dummy IdentityProviderHandler to fix integration tests

* Replace KeyValueStore with KeyValueStorage (#663)

* feat: Create WrappedExpiringStorage

* refactor: Update ResourceStoreEmailPasswordStore to use KeyValueStorage

* refactor: Update KeyGeneratingIdpConfigurationGenerator to use KeyValueStorage

* refactor: Update ResourceStoreStorageAdapterFactory to use ExpiringStorage

* refactor: Removed KeyValueStore

* refactor: Simplify EmailPassword handlers (#664)

* refactor: Order index.ts

* test: Add EmailPasswordForgotPasswordHandler unit tests

* test: Add EmailPasswordGetResetPasswordHandler unit tests

* test: Add EmailPasswordLoginHandler unit tests

* test: Add EmailPasswordRegistrationHandler unit tests

* test: Add EmailPasswordResetPasswordHandler unit tests

* test: Remove unnecessary test file

* feat: Basic instructions for using the IdP

* fix: IdP instructions and add example WebID

* fix: IdP registration copy

* fix: IdP instruction editorial

* Update README.md

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* Update README.md

Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>

* test: Add KeyGeneratingIdpConfigurationGenerator unit tests

* test: Add KeyValueEmailPasswordStore unit tests

* test: Create IdP integration test

* test: Add EmailPasswordInteractionPolicy unit tests

* test: Add BasicIssuerReferenceWebIdOwnershipValidator unit tests

* test: Add ChooseInitialInteractionHandler unit tests

Also fixes the config warning.

* test: Add EjsTemplateRenderer unit tests

* test: Add EmailSender unit tests

* test: Add FormDataUtil unit tests

* test: Add IdpRouteController unit tests

* test: Add OidcInteractionCompleter unit tests

* refactor: Simplify ClientWebIdFetchingStorageAdapterFactory

* test: Add ClientWebIdFetchingStorageAdapterFactory unit tests

* refactor: Fix ejs html warnings

* test: Add step to test logging in again

Included are updates to handle cookies more correctly.

* feat: Add IdpConfirmHttpHandler

This way there's a handler for the confirm page.

* test: Add ExpiringStorageAdapterFactory unit tests

* test: Add IdentityProviderFactory unit tests

* test: Add IdentityProviderHttpHandler unit tests

* refactor: Minor refactors

* refactor: Use jose instead of node-jose

* refactor: Use jose instead of node-jose

Reduces the number of dependencies since other libraries
also depend on jose.

* Update src/identity/configuration/KeyGeneratingIdpConfigurationGenerator.ts

Co-authored-by: Matthieu Bosquet <matthieubosquet@gmail.com>

* refactor: Use interfaces instead of abstract classes

* refactor: Make WebIdOwnershipValidator an AsyncHandler

* refactor: Make TemplateRenderer an AsyncHandler

* fix: Fix typing issue

* fix: Convert JWK to plain object for node 15

* feat: Update CI configuration

--ignore-scripts was removed because it also stopped dependency scripts,
which was a requirement for bcrypt.
15.0 was removed since that version doesn't run the required scripts after install.
14.0 was removed since the somehow it caused the solid-authn client to do the wrong calls.

* test: Run integration tests on Node 14.2

This is the lowest 14.x version where the IdP integration tests succeed.

* feat: Use ErrorResponseWriter for handling oidc errors

* test: Mock Date in OidcInteractionCompleter tests

* fix: Correctly generate new identifiers

Previously there could be double slashes if the base URL ended in slash.

* fix: Correctly handle storagePathName in ExpiringStorageAdapterFactory

* fix: Fix issue with new CliRunner test in rebase

* fix: Handle unknown errors more consistently

* feat: Make idp parameter dynamic

* feat: Add more logging

* refactor: Link css instead of injecting

* fix: Fix redis integration tests with idp

* refactor: Shorten idp class names

* refactor: Remove e-mail configuration from default config

* feat: Store JsonResourceStorage data in a single container

* feat: Make sure expired data gets removed at some point

* feat: Only accept strings as keys in KeyValueStorage

* fix: Various minor fixes based on review

Co-authored-by: Matthieu Bosquet <matthieubosquet@gmail.com>
Co-authored-by: Joachim Van Herwegen <joachimvh@gmail.com>
2021-05-04 13:17:43 +02:00
Joachim Van Herwegen
6cd2806c2f test: Remove node-mocks-http references from integration tests 2021-05-04 11:27:13 +02:00
Joachim Van Herwegen
4083d24e4a test: Update LdpHandlerWitAuth to use fetch for tests
The AclHelper class has been extended to be more flexible for future tests.
2021-05-04 11:27:13 +02:00
Joachim Van Herwegen
dd3fb63d18 test: Update LdpHandlerWithoutAuth to use fetch for tests 2021-05-04 11:27:13 +02:00
Joachim Van Herwegen
fa8d406f34 test: Create single function for port generation
This reduces the chances of several integration tests
re-using the same port.
2021-05-04 11:27:13 +02:00
Joachim Van Herwegen
6288003915 feat: Create KeyValueStorage with a JSON file backend 2021-03-02 11:47:07 +01:00
Ruben Verborgh
a57105be8e change: Query string does not influence identifier. 2021-01-29 09:12:49 +01:00
Ruben Verborgh
f9a20799eb refactor: Use declarations style for functions. 2021-01-06 11:24:43 +01:00
Ruben Verborgh
af8524e0d6 test: Make helper terminology consistent. 2020-12-22 01:02:19 +01:00
Joachim Van Herwegen
f387b36dc2 feat: add template based data generator 2020-12-02 13:01:38 +01:00
Joachim Van Herwegen
e00cb05dc3 feat: Add support for mocking fs 2020-10-14 11:32:27 +02:00
Joachim Van Herwegen
9657fbafb1 refactor: Update eslint related dependencies 2020-09-17 11:55:49 +02:00
Joachim Van Herwegen
b1991cb08a
feat: More integration tests and test configs (#154)
* add todo

* Added configurations folder

* Added config to index

* changed /bin/server to use configfiles

* initiate acl with Setup component

* add last changes

* reset serverconfig

* move authenticatedLdpHandler configs to config files

* failed to read testss

* failed to read testss

* removed import part

* Fix merge conflicts

* add first FileResTests

* fix: fix fileresourcestore metadata error

* fix unit tests

* remove test files

* Added test and changed callFile

* Fix: metadata file error in FileResourceStore

* fix: ensure full test coverage

* added tests

* Fix get tests

* added testfiles

* changed config to use PasstrueStore

* to continue work on

* refactor fileresourcestore config

* refactor tests

* fix content-types, update tests

* replace sync methods with async methods

* move acl function to util

* added testfiles for Fileserver with acl

* update tests

* add first acl filestore test

* refactor

* add resource mapper

* refactor config files

* add more fileresourcestore acl tests

* add locking resource store test files

* move file mapping logic to resourcemapper

* added beforeAll for a permanent file in Auht tests

* make filestore dependent of resource mapper

* moved configs to test/configs

* set default contenttype

* refactor fileresourcemapper

* fix map function

* refactor

* fixed foldercreationtest

* changed some tests so the files are cleaned up when done testing

* add normalized parser

* Auhtenticationtests clean up acl file

* refactor unit test

* lost changes added again

* fix metadata problem

* refactor names

* reverse change

* add getters

* configs and start util

* add comments

* add comments, move code

* added acl helper and changed tests

* linter 7.7.0 -> 7.0.0

* moved test/tesfiles -> test/assets

* removed configs/**/*.ts from tsconfig.json

* Temporary changed threshold so cli test is ignored and commiting goes easier, will revert later

* added FileResourceStore to index

* Changed imports

* Changed imports for all configs

* Removed comment

* Changed names of configs

* added 'Config' to name and removed comment

* removed unused testfile and added testfile 0

* changed beforeAll to just copy permanent file

* change text/turtle to constant

* fix converter issue

* getHandler -> getHttpHandler, and updates to config

* removed ','

* removed trailing /

* changed imports for index.d.js problem

* removed duplicate file and added line that got removed in mergeconflicts

* add jest global teardown

* add ignore for CliRunner

* add changes

* fix copyfile error

* remove unused testfiles

* adding test with image

* add first util functions

* relative paths to absolute paths

* added 3 FileStoreTests

* more refactoring

* more absolute paths

* fix mkdir path

* added test

* add util for easy configs

* add comments

* added some testhelpers and refactor first test

* fix converter test error

* refactor FileResTests

* solved failing test because new converters

* removed afterAll()

* removed setAcl from util

* removed config from Authorization.test.ts

* changed strange linting

* refactored AuthenticatedFileResourceStore tests

* fixed unclear root variable

* fix: Use absolute test paths

* Mock fs correctly and remove teardown

* Clean up after tests

Co-authored-by: freyavs <freyavanspeybroeck@outlook.com>
Co-authored-by: thdossch <dossche.thor@gmail.com>
Co-authored-by: Freya <56410697+freyavs@users.noreply.github.com>
Co-authored-by: thdossch <49074469+thdossch@users.noreply.github.com>
Co-authored-by: Ruben Verborgh <ruben@verborgh.org>
2020-09-14 16:06:27 +02:00
Ruben Taelman
48740e5cba style: improve linting of imports
A new plugin is used that offers autofixing capabilities.
2020-08-26 13:20:19 +02:00
Joachim Van Herwegen
769b49293c feat: Integrate acl with rest of server 2020-08-18 10:32:07 +02:00