Wout Slabbinck ca62511d12 feat: Edit metadata resources (#1188)
* feat: only allow metadata to be edited with PATCH request & only allow metadata files to be edited when a resource is available

* fix: remove unnecesary log at POST

* feat: PUT resets metadata contents + not possible to add metadata with PUT to container

* feat: add metadataStrategy (auxiliaryStrategy) + use that strategy in operationhandlers

* feat: PUT request on existing LDPC is not allowed as it would be possible to edit (read reset) metadata

* test: add unit tests to operationhandlers to handle metadata editing

* test: add unit tests to representationPatchHandler to handle metadata editing

* fix: update dependency of meta.json to version 3.0.0

* fix: lint and dependency still v2

* fix: replaced file references to resource references + moved Patch check to new patchhandler which is more generic

* fix: moved checking metadata resources checking from DELETE and POST handler down to DABS

* fix: remove PATCH message about metadata extension

* fix: move PATCH message about metadata extension

* WIP: adding writeMetadata + getMetadata in DABS and add writeMetadata to DataAccessors (part 1)

* WIP: implement writeMetadata in memorybackend + change resourceExists

* WIP: implement writeMetadata in SparqlDataAccessor.ts

* test: fix test interfaces

* test: InMemoryDataAccessor.ts resulted into changing identifier for writeMetadata in DataAccessor.ts (now taking subject identifier instead of metadata resource identifier)

* test: accessor tests implemented for metadata

* test: add RdfImmutableCheckPatcher.test.ts

* test: add tests in DataAccessorBasedStore.test.ts

* test: fix template config for DynamicPods test

* test: add integration tests for metadata

* fix: change metaStrategy to metadataStrategy

* refactor: comments updated to new location CSS on github + some alphabetical edits

* refactor: remove getMetadata function in DABS as it is only used once

* refactor: add DataAccessorBasedStoreArgs to DataAccessorBasedStore.ts

* docs: modify documentation for writeMetadata function in DataAccessor.ts

* feat: ldp:contains is also part of the metadata resource of a container

* refactor: change function name and move check to DataAccessorBasedStore

* fix: fix tests for DABS and PutOperationHandler

* feat: avoid cloneRepresentation by introducing RdfPatcher, RdfStorePatcher and modifying ImmutableMetadataPatcher, N3Patcher, patching.json and SparqlUpdatePatcher

* test: fix patcher tests

* feat: create sparqlInsertMetadata in SparqlDataAccessor.ts

* fix: move check during put on container if it exists already back to PutOperationHandler.ts after discussion in PR

* test: update tests PutOperationHandler.ts and DataAccessorBasedStore.ts regarding previous commit

* test: add converter to DABS and replace rejection on data during container creation to warning

* test: implemented RdfPatcher test

* feat: remove ContainerPatcher

* fix: fix lint

* fix: fix integration tests

* refactor: fix minor issues mentioned in the PR

* WIP: problem with removeResponseMetadata

* refactor: remove responseMetadata in QuadToRdfConverter.ts

* feat: handle ResponeMetadata when writing to the store via a patch

* refactor: refactor based on comments in PR

* feat: make ImmutableMetadataPatcher.ts instantiation more clear

* test: achieve 100% coverage again

* fix: fix lint

* refactor: return to explicit arguments for the DABS

* fix: return to explicit arguments for the DABS (missed one)

* feat: optimise immutable checker

* fix: fix, enhance docs + optimise config files

* fix: DABS + QuadToRdfConverter feedback implemented

* fix: patching feedback implemented

* test: update operationhandler tests

* test: update integration tests after feedback

* test: update DABS tests after feedback

* test: update ImmutableMetadataPatcher.test.ts after feedback

* test: update patch tests after feedback

* docs: add documentation about editing metadata

* fix: config: intendation + name change + extra filters | filter pattern

* docs: tsdoc added to RdfStorePatcher.ts

* fix: DABS split implemented for getRepresentation + comment refactoring

* docs: further documentation on removing response data on serialization

* fix: DABS getRepresentation method

* docs: apply feedback from Joachim on the documentation of metadata-editing.md

* fix: indentation fix + fix metadata-editing.md documentation after feedback from Joachim

* docs: small fix in metadata-editing.md documentation after feedback from Joachim

* fix: fix metadata-editing.md documentation after feedback from Joachim

* fix: fix tests meta-editing after feedback Joachim

* feat: first attempt at RELEASE_NOTES.md

* docs: update release notes based on feedback

* docs: fix newline

* fix: patching config changes after feedback

* docs: metadata editing documentation changes after feedback

* docs: metadata editing documentation changes after feedback

* docs: metadata editing documentation changes after feedback

* feat: optimisation on ImmutableMetadataPatcher.ts algorithm

* feat: remove converter from DABS and add conversion for metadata resources in the RCS

* fix: Fix documentation RepresentationPatchHandler.ts + fix response graph not being stored due to convertingstore

* feat: make RepresentationPatcher generic

* test: generic RepresentationPatcher tests

* test: 100% coverage for patchers again

* feat: containers can be created with POST with no content-type

* feat: Immutable checks always with subject identifier

* feat: create AuxiliaryLinkMetadataWriter for adding description resources Link Header

* test: add tests for AuxiliaryLinkMetadataWriter and update them for ImmutableMetadataPatcher

* feat: remove metadataGenerator from acl.json and fix tests accordingly

* WIP: preserve metadata on PUT

* feat: preserve metadata on PUT

* fix: keep metadata on PATCHes

* test: add unit tests for preserving metadata on PUT

* fix: remove inConverter from sparql endpoint as that is already the default in the (converting.json)

* fix: add metadatastrategy to RepresentationConvertingStore in regex.json

* test: add integration tests for preserving metadata on PUT

* docs: update release notes and adding documentation about preserving metadata on PUT

* WIP: Template create setRepresentation

* fix: Move container exists and not allowed check to setRepresentation

* test: fix lint

* fix: update configs and documentation

* refactor: update and add documentation + small refactoring

* refactor: update and add documentation + small refactoring + fix tests

* fix: Dynamic pod config + tests

* fix: TemplatedResourcesGenerator does not create containers when they already exist

* fix: metadata preservation now deals with complex content types

* docs: explain the case when there is no content-type

* fix: minor comments
2022-08-04 15:24:36 +02:00
2022-08-04 15:24:36 +02:00
2021-10-21 13:57:28 +01:00
2021-07-16 14:32:42 +01:00
2022-06-16 11:10:39 +02:00
2022-04-21 15:45:31 +02:00

Community Solid Server

[Solid logo]

MIT license npm version Node.js version Build Status Coverage Status GitHub discussions Chat on Gitter

The Community Solid Server is open software that provides you with a Solid Pod and identity. This Pod acts as your own personal storage space so you can share data with people and Solid applications.

As an open and modular implementation of the Solid specifications, the Community Solid Server is a great companion:

  • 🧑🏽 for people who want to try out having their own Pod

  • 👨🏿‍💻 for developers who want to create and test Solid apps

  • 👩🏻‍🔬 for researchers who want to design new features for Solid

And, of course, for many others who like to experience Solid.

You can install the software locally or on your server and get started with Solid immediately.

Running the server

To run the server, you will need Node.js. We support versions 14.2 and up.
If you do not use Node.js, you can run a Docker version instead.

💻 Installing and running locally

After installing Node.js, install the latest server version from the npm package repository:

npm install -g @solid/community-server

To run the server with in-memory storage, use:

community-solid-server # add parameters if needed

To run the server with your current folder as storage, use:

community-solid-server -c @css:config/file.json

📃 Installing and running from source

If you rather prefer to run the latest source code version, or if you want to try a specific branch of the code, you can use:

git clone https://github.com/CommunitySolidServer/CommunitySolidServer.git
cd CommunitySolidServer
npm ci
npm start -- # add parameters if needed

📦 Running via Docker

Docker allows you to run the server without having Node.js installed. Images are built on each tagged version and hosted on Docker Hub.

# Clone the repo to get access to the configs
git clone https://github.com/CommunitySolidServer/CommunitySolidServer.git
cd CommunitySolidServer
# Run the image, serving your `~/Solid` directory on `http://localhost:3000`
docker run --rm -v ~/Solid:/data -p 3000:3000 -it solidproject/community-server:latest
# Or use one of the built-in configurations
docker run --rm -p 3000:3000 -it solidproject/community-server -c config/default.json
# Or use your own configuration mapped to the right directory
docker run --rm -v ~/solid-config:/config -p 3000:3000 -it solidproject/community-server -c /config/my-config.json

🗃️ Helm Chart

The official Helm Chart for Kubernetes deployment is maintained at CommunitySolidServer/css-helm-chart and published on ArtifactHUB. There you will find complete installation instructions.

# Summary
helm repo add community-solid-server https://communitysolidserver.github.io/css-helm-chart/charts/
helm install my-css community-solid-server/community-solid-server

🔧 Configuring the server

The Community Solid Server is designed to be flexible such that people can easily run different configurations. This is useful for customizing the server with plugins, testing applications in different setups, or developing new parts for the server without needing to change its base code.

⏱️ Parameters

An easy way to customize the server is by passing parameters to the server command. These parameters give you direct access to some commonly used settings:

parameter name default value description
--port, -p 3000 The TCP port on which the server should listen.
--baseUrl, -b http://localhost:$PORT/ The base URL used internally to generate URLs. Change this if your server does not run on http://localhost:$PORT/.
--loggingLevel, -l info The detail level of logging; useful for debugging problems. Use debug for full information.
--config, -c @css:config/default.json The configuration for the server. The default only stores data in memory; to persist to your filesystem, use @css:config/file.json
--rootFilePath, -f ./ Root folder where the server stores data, when using a file-based configuration.
--sparqlEndpoint, -s URL of the SPARQL endpoint, when using a quadstore-based configuration.
--showStackTrace, -t false Enables detailed logging on error output.
--podConfigJson ./pod-config.json Path to the file that keeps track of dynamic Pod configurations. Only relevant when using @css:config/dynamic.json.
--seededPodConfigJson Path to the file that keeps track of seeded Pod configurations.
--mainModulePath, -m Path from where Components.js will start its lookup when initializing configurations.
--workers, -w 1 Run in multithreaded mode using workers. Special values are -1 (scale to num_cores-1), 0 (scale to num_cores) and 1 (singlethreaded).

🔀 Multithreading

The Community Solid Server can be started in multithreaded mode with any config. The config must only contain components that are threadsafe though. If a non-threadsafe component is used in multithreaded mode, the server will describe with an error which class is the culprit.

# Running multithreaded with autoscaling to number of logical cores minus 1
npm start -- -c config/file.json -w -1

🧶 Custom configurations

More substantial changes to server behavior can be achieved by writing new configuration files in JSON-LD. The Community Solid Server uses Components.js to specify how modules and components need to be wired together at runtime.

Examples and guidance on configurations are available in the config folder.

Recipes for configuring the server can be found at CommunitySolidServer/recipes.

👩🏽‍💻 Developing server code

The server allows writing and plugging in custom modules without altering its base source code.

The 📗 API documentation and the 📐 architectural diagram can help you find your way.

If you want to help out with server development, have a look at the 📓 user documentation and 🛠️ good first issues.

📜 License

The Solid Community Server code is copyrighted by Inrupt Inc. and imec and available under the MIT License.

Core contributors are Joachim Van Herwegen, Ruben Verborgh, Ruben Taelman, and Matthieu Bosquet.

🎤 Feedback and questions

Don't hesitate to start a discussion or report a bug.

Learn more about Solid at solidproject.org.

Description
An open and modular implementation of the Solid specifications
Readme 72 MiB
Languages
TypeScript 97.6%
EJS 1%
HTML 0.4%
JavaScript 0.4%
Handlebars 0.2%
Other 0.4%