Compare commits

..

38 Commits

Author SHA1 Message Date
vorotamoroz
2f10121d6c Merge remote-tracking branch 'origin/main' into cli_test_deno 2026-05-22 10:19:30 +00:00
vorotamoroz
1b8747115c Merge pull request #922 from vrtmrz/update_trystero
Releasing v0.25.68 (dependency and P2P update)
2026-05-22 19:17:22 +09:00
vorotamoroz
e739302fb9 bump 2026-05-22 11:11:18 +01:00
vorotamoroz
8f20d53f55 Update trystero, typed well. fixed some potentially problems
Weaken terser for libraries
2026-05-22 11:01:51 +01:00
vorotamoroz
3ab80190d6 test fix ci (Redundant test) 2026-05-22 03:48:41 +00:00
vorotamoroz
8948bf2803 test cli:p2p use nonce for peername 2026-05-22 03:48:02 +00:00
vorotamoroz
486fd15c60 fix resouce handling 2026-05-22 03:46:56 +00:00
vorotamoroz
5fd85c71ca test: chore: prettify 2026-05-22 03:20:28 +00:00
vorotamoroz
c1f41910c4 test: add actions / caching 2026-05-22 03:20:11 +00:00
vorotamoroz
3693d6a6b6 test: add port ready, container cleanup 2026-05-22 03:19:48 +00:00
vorotamoroz
cc3c992b1d cli: add large-file-test and benchmark between couchdb and p2p 2026-05-22 03:05:44 +00:00
vorotamoroz
df390ac456 test: fix deno test helpers 2026-05-22 03:02:11 +00:00
vorotamoroz
fd84b0377b Update actions 2026-05-21 10:02:46 +01:00
vorotamoroz
340d416b76 Merge pull request #920 from vrtmrz/v0_25_66
Release v0.25.67
2026-05-20 19:44:41 +09:00
vorotamoroz
3034af8d69 bump again 2026-05-20 11:37:51 +01:00
vorotamoroz
da3020bd45 fixed: fix auto-correction mismatch 2026-05-20 11:34:28 +01:00
vorotamoroz
ce232c1002 bump 2026-05-20 11:14:41 +01:00
vorotamoroz
0e13926400 fixed: update lib
I should do this with #919
2026-05-20 11:10:43 +01:00
vorotamoroz
fab7ec996a Merge pull request #919 from vrtmrz:feat_cli_remote_select
feat: add CLI commands to handle multiple remote configuration
2026-05-20 19:05:49 +09:00
vorotamoroz
88e22f99c5 Merge pull request #917 from vrtmrz:feat_tweak_auto_adjust
feat: implement auto-accept compatible tweak
2026-05-20 19:02:58 +09:00
vorotamoroz
83cbabf06f Merge branch 'main' into feat_tweak_auto_adjust 2026-05-20 11:01:51 +01:00
vorotamoroz
5e8d3b8f02 Update lib 2026-05-20 11:00:38 +01:00
vorotamoroz
1167b41340 feat: add CLI commands to handle multiple remote configuration 2026-05-20 05:10:42 +01:00
vorotamoroz
67da3964e5 Merge pull request #918 from vrtmrz/v0_25_65
V0.25.65
2026-05-19 20:27:34 +09:00
vorotamoroz
45ebc7eb6b Update the doc and issue report, and bump 2026-05-19 12:17:26 +01:00
vorotamoroz
cc5ead68bc minor layout fix 2026-05-19 11:59:54 +01:00
vorotamoroz
9b9e4f22f3 Merge pull request #916 from vrtmrz:fix_866
fixed: Now Chunk Splitter: `V3: Fine Deduplication`
2026-05-19 19:46:08 +09:00
vorotamoroz
7823f46053 Merge branch 'main' into fix_866 2026-05-19 11:45:41 +01:00
vorotamoroz
d6d8e548b3 Merge pull request #915 from vrtmrz/improve_log_feature
Improve log features
2026-05-19 19:40:28 +09:00
vorotamoroz
44b1ed7610 Merge branch 'main' into improve_log_feature 2026-05-19 11:40:14 +01:00
vorotamoroz
5786da5534 merged 2026-05-19 11:38:05 +01:00
vorotamoroz
042a80dd44 Merge pull request #914 from vrtmrz/fix_ios_resume
fix: Fix an issue about resuming from background on iOS (#888).
2026-05-19 19:34:49 +09:00
vorotamoroz
ee30f6cd6d fixed: Now Chunk Splitter: V3: Fine Deduplication is working fine again (#866). 2026-05-19 11:26:35 +01:00
vorotamoroz
977a300808 fix grammatical error 2026-05-19 04:10:39 +01:00
vorotamoroz
a392ccab6a - Improved an error verbosity on concurrent processing on start-up process.
- Now the `report` includes recent logs (of verbosity `verbose` even settings is not set to `verbose`).
- Updating logs is now debounced to avoid excessive updates during rapid log generation.
- Added a `Generate full report for opening the issue with debug info` command to the command palette, which generates a report without opening the settings dialogue.
2026-05-19 04:09:04 +01:00
vorotamoroz
dfdfa5383b grammatical fix 2026-05-19 01:55:10 +01:00
vorotamoroz
a08294ab16 fix: Fix an issue about resuming from background on iOS (#888). 2026-05-18 12:30:03 +01:00
vorotamoroz
e8c33a0d6a feat: implement auto-accept compatible tweak setting and enhance mismatch resolution logic 2026-05-18 11:21:53 +01:00
40 changed files with 2636 additions and 520 deletions

View File

@@ -53,11 +53,12 @@ The hatch report (below) includes version information. If you cannot provide the
- Self-hosted LiveSync version: <!-- e.g. 0.23.0 — find it in Obsidian Settings → Community Plugins -->
### Report from LiveSync
Open the `Hatch` pane in LiveSync settings and press `Make report`. Paste here or upload to [Gist](https://gist.github.com/) and share the link.
### Report and Logs from LiveSync
Perform a `Generate full report for opening the issue with debug info` command and provide the generated report. This contains detailed information and recent 1000 log lines, which is very helpful for debugging. **PLEASE AMEND THE REPORT TO REMOVE ANY SENSITIVE INFORMATION BEFORE PASTING.**
If too large to paste here, upload to [Gist](https://gist.github.com/) and share the link.
<details>
<summary>Report from hatch (primary)</summary>
<summary>Report and Logs (primary)</summary>
```
<!-- paste here or link to Gist -->
@@ -65,29 +66,7 @@ Open the `Hatch` pane in LiveSync settings and press `Make report`. Paste here o
</details>
<details>
<summary>Report from hatch (if applicable)</summary>
```
<!-- paste here or link to Gist -->
```
</details>
### Plug-in log
Enable `Verbose Log` in General Settings first, then reproduce the issue and copy the log (tap the document box icon in the ribbon).
Paste here or upload to [Gist](https://gist.github.com/) and share the link.
<details>
<summary>Plug-in log (primary)</summary>
```
<!-- paste here or link to Gist -->
```
</details>
<details>
<summary>Plug-in log (if applicable)</summary>
<summary>Report and Logs (if applicable)</summary>
```
<!-- paste here or link to Gist -->

View File

@@ -32,13 +32,13 @@ jobs:
case "$SELECTED_TASK" in
test)
TASK_MATRIX='["test:setup-put-cat","test:mirror","test:push-pull","test:sync-two-local","test:sync-locked-remote","test:p2p-host","test:p2p-peers","test:p2p-sync","test:p2p-three-nodes","test:p2p-upload-download","test:e2e-couchdb","test:e2e-matrix"]'
TASK_MATRIX='["test:setup-put-cat","test:mirror","test:push-pull","test:sync-two-local","test:sync-locked-remote","test:p2p-host","test:p2p-peers","test:p2p-sync","test:p2p-three-nodes","test:p2p-upload-download","test:e2e-matrix:couchdb-enc0","test:e2e-matrix:couchdb-enc1","test:e2e-matrix:minio-enc0","test:e2e-matrix:minio-enc1"]'
;;
test:local)
TASK_MATRIX='["test:setup-put-cat","test:mirror"]'
;;
test:e2e-matrix)
TASK_MATRIX='["test:e2e-matrix"]'
TASK_MATRIX='["test:e2e-matrix:couchdb-enc0","test:e2e-matrix:couchdb-enc1","test:e2e-matrix:minio-enc0","test:e2e-matrix:minio-enc1"]'
;;
test:p2p-sync)
TASK_MATRIX='["test:p2p-sync"]'
@@ -55,6 +55,8 @@ jobs:
needs: prepare
runs-on: ubuntu-latest
timeout-minutes: 60
env:
DENO_DIR: ~/.cache/deno
strategy:
fail-fast: false
matrix:
@@ -70,12 +72,21 @@ jobs:
with:
node-version: '24.x'
cache: 'npm'
cache-dependency-path: package-lock.json
- name: Setup Deno
uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- name: Cache Deno dependencies
uses: actions/cache@v4
with:
path: ~/.cache/deno
key: ${{ runner.os }}-deno-${{ hashFiles('src/apps/cli/testdeno/deno.lock', 'src/apps/cli/testdeno/deno.json') }}
restore-keys: |
${{ runner.os }}-deno-
- name: Install dependencies
run: npm ci

View File

@@ -9,6 +9,10 @@ on:
jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: write
id-token: write
attestations: write
steps:
- uses: actions/checkout@v4
with:
@@ -29,68 +33,20 @@ jobs:
run: |
npm ci
npm run build --if-present
# Attest
- name: Attest Plugin Artifacts
uses: actions/attest-build-provenance@v4
with:
subject-path: |
main.js
manifest.json
styles.css
# Package the required files into a zip
- name: Package
run: |
mkdir ${{ github.event.repository.name }}
cp main.js manifest.json styles.css README.md ${{ github.event.repository.name }}
zip -r ${{ github.event.repository.name }}.zip ${{ github.event.repository.name }}
# Create the release on github
# - name: Create Release
# id: create_release
# uses: actions/create-release@v1
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# VERSION: ${{ steps.version.outputs.tag }}
# with:
# tag_name: ${{ steps.version.outputs.tag }}
# release_name: ${{ steps.version.outputs.tag }}
# draft: true
# prerelease: false
# # Upload the packaged release file
# - name: Upload zip file
# id: upload-zip
# uses: actions/upload-release-asset@v1
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.create_release.outputs.upload_url }}
# asset_path: ./${{ github.event.repository.name }}.zip
# asset_name: ${{ github.event.repository.name }}-${{ steps.version.outputs.tag }}.zip
# asset_content_type: application/zip
# # Upload the main.js
# - name: Upload main.js
# id: upload-main
# uses: actions/upload-release-asset@v1
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.create_release.outputs.upload_url }}
# asset_path: ./main.js
# asset_name: main.js
# asset_content_type: text/javascript
# # Upload the manifest.json
# - name: Upload manifest.json
# id: upload-manifest
# uses: actions/upload-release-asset@v1
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.create_release.outputs.upload_url }}
# asset_path: ./manifest.json
# asset_name: manifest.json
# asset_content_type: application/json
# # Upload the style.css
# - name: Upload styles.css
# id: upload-css
# uses: actions/upload-release-asset@v1
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.create_release.outputs.upload_url }}
# asset_path: ./styles.css
# asset_name: styles.css
# asset_content_type: text/css
- name: Create Release and Upload Assets
uses: softprops/action-gh-release@v2
with:

View File

@@ -10,7 +10,18 @@ on:
paths:
- 'src/**'
- 'test/**'
- 'lib/**'
- 'package.json'
- 'package-lock.json'
- 'tsconfig.json'
- 'vite.config.ts'
- 'vitest.config*.ts'
- 'esbuild.config.mjs'
- 'eslint.config.mjs'
- '.github/workflows/unit-ci.yml'
pull_request:
paths:
- 'src/**'
- 'test/**'
- 'package.json'
- 'package-lock.json'
- 'tsconfig.json'

4
.gitignore vendored
View File

@@ -28,4 +28,6 @@ data.json
cov_profile/**
coverage
src/apps/cli/dist/*
src/apps/cli/dist/*
_testdata/**
utils/bench/splitResults.csv

View File

@@ -255,14 +255,20 @@ It depends on Obsidian detects. May toggling `Detect all extensions` of
### I hope to report the issue, but you said you needs `Report`. How to make it?
We can copy the report to the clipboard, by pressing the `Make report` button on
the `Hatch` pane. ![Screenshot](../images/hatch.png)
We can copy the report to the clipboard, by performing
`Generate full report for opening the issue with debug info` command!
### Where can I check the log?
We can launch the log pane by `Show log` on the command palette. And if you have
troubled something, please enable the `Verbose Log` on the `General Setting`
pane.
`Generate full report for opening the issue with debug info` command also contains
the recent 1000 log lines, which is very helpful for debugging. Full-report is
already set to the verbose level, so it contains all the logs without enabling the
`Verbose Log` toggle.
Let me note that please be sure to remove any sensitive information before sharing the report.
However, the logs would not be kept so long and cleared when restarted. If you
want to check the logs, please enable `Write logs into the file` temporarily.

View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.25.64",
"version": "0.25.68",
"minAppVersion": "1.7.2",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

290
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "obsidian-livesync",
"version": "0.25.64",
"version": "0.25.68",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "0.25.64",
"version": "0.25.68",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
@@ -15,7 +15,7 @@
"@smithy/middleware-apply-body-checksum": "^4.3.9",
"@smithy/protocol-http": "^5.3.9",
"@smithy/querystring-builder": "^4.2.9",
"@trystero-p2p/nostr": "^0.23.0",
"@trystero-p2p/nostr": "^0.24.0",
"chokidar": "^4.0.0",
"commander": "^14.0.3",
"diff-match-patch": "^1.0.5",
@@ -28,7 +28,7 @@
"octagonal-wheels": "^0.1.45",
"pouchdb-adapter-leveldb": "^9.0.0",
"qrcode-generator": "^1.4.4",
"werift": "^0.22.9",
"werift": "^0.23.0",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
},
"devDependencies": {
@@ -1851,9 +1851,9 @@
"license": "MIT"
},
"node_modules/@eslint/config-array/node_modules/brace-expansion": {
"version": "1.1.13",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.13.tgz",
"integrity": "sha512-9ZLprWS6EENmhEOpjCYW2c8VkmOvckIJZfkr7rBW6dObmfgJ/L1GpSYW5Hpo9lDz4D1+n0Ckz8rU7FwHDQiG/w==",
"version": "1.1.14",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.14.tgz",
"integrity": "sha512-MWPGfDxnyzKU7rNOW9SP/c50vi3xrmrua/+6hfPbCS2ABNWfx24vPidzvC7krjU/RTo235sV776ymlsMtGKj8g==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -1887,7 +1887,7 @@
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
}
},
"node_modules/@eslint/config-helpers/node_modules/@eslint/core": {
"node_modules/@eslint/core": {
"version": "0.17.0",
"resolved": "https://registry.npmjs.org/@eslint/core/-/core-0.17.0.tgz",
"integrity": "sha512-yL/sLrpmtDaFEiUj1osRP4TI2MDz1AddJL+jZ7KSqvBuliN4xqYY54IfdN8qD8Toa6g1iloph1fxQNkjOxrrpQ==",
@@ -1932,9 +1932,9 @@
"license": "MIT"
},
"node_modules/@eslint/eslintrc/node_modules/brace-expansion": {
"version": "1.1.13",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.13.tgz",
"integrity": "sha512-9ZLprWS6EENmhEOpjCYW2c8VkmOvckIJZfkr7rBW6dObmfgJ/L1GpSYW5Hpo9lDz4D1+n0Ckz8rU7FwHDQiG/w==",
"version": "1.1.14",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.14.tgz",
"integrity": "sha512-MWPGfDxnyzKU7rNOW9SP/c50vi3xrmrua/+6hfPbCS2ABNWfx24vPidzvC7krjU/RTo235sV776ymlsMtGKj8g==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -1984,19 +1984,6 @@
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
}
},
"node_modules/@eslint/json/node_modules/@eslint/core": {
"version": "0.17.0",
"resolved": "https://registry.npmjs.org/@eslint/core/-/core-0.17.0.tgz",
"integrity": "sha512-yL/sLrpmtDaFEiUj1osRP4TI2MDz1AddJL+jZ7KSqvBuliN4xqYY54IfdN8qD8Toa6g1iloph1fxQNkjOxrrpQ==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"@types/json-schema": "^7.0.15"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
}
},
"node_modules/@eslint/object-schema": {
"version": "2.1.7",
"resolved": "https://registry.npmjs.org/@eslint/object-schema/-/object-schema-2.1.7.tgz",
@@ -2021,19 +2008,6 @@
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
}
},
"node_modules/@eslint/plugin-kit/node_modules/@eslint/core": {
"version": "0.17.0",
"resolved": "https://registry.npmjs.org/@eslint/core/-/core-0.17.0.tgz",
"integrity": "sha512-yL/sLrpmtDaFEiUj1osRP4TI2MDz1AddJL+jZ7KSqvBuliN4xqYY54IfdN8qD8Toa6g1iloph1fxQNkjOxrrpQ==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"@types/json-schema": "^7.0.15"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
}
},
"node_modules/@fidm/asn1": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/@fidm/asn1/-/asn1-1.0.4.tgz",
@@ -2710,9 +2684,9 @@
}
},
"node_modules/@noble/secp256k1": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/@noble/secp256k1/-/secp256k1-3.0.0.tgz",
"integrity": "sha512-NJBaR352KyIvj3t6sgT/+7xrNyF9Xk9QlLSIqUGVUYlsnDTAUqY8LOmwpcgEx4AMJXRITQ5XEVHD+mMaPfr3mg==",
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/@noble/secp256k1/-/secp256k1-3.1.0.tgz",
"integrity": "sha512-+F7iS7tUMaNGXcc9X3PjmjvuQnXEuSjCRNzVVA2xAcKXgCaP0dHYz4SFyt4FKNHef7sOP//xihowcySSS7PK9g==",
"license": "MIT",
"funding": {
"url": "https://paulmillr.com/funding/"
@@ -3572,20 +3546,13 @@
}
},
"node_modules/@smithy/core": {
"version": "3.23.12",
"resolved": "https://registry.npmjs.org/@smithy/core/-/core-3.23.12.tgz",
"integrity": "sha512-o9VycsYNtgC+Dy3I0yrwCqv9CWicDnke0L7EVOrZtJpjb2t0EjaEofmMrYc0T1Kn3yk32zm6cspxF9u9Bj7e5w==",
"version": "3.24.3",
"resolved": "https://registry.npmjs.org/@smithy/core/-/core-3.24.3.tgz",
"integrity": "sha512-Ep/7tPamGY8mgESE3LyLKtxJyy6U52WWAqr/3wial47Sj4u3PiIF73AOGI27UyLy9duTkhZbgzodOfLV4TduZg==",
"license": "Apache-2.0",
"dependencies": {
"@smithy/protocol-http": "^5.3.12",
"@smithy/types": "^4.13.1",
"@smithy/url-parser": "^4.2.12",
"@smithy/util-base64": "^4.3.2",
"@smithy/util-body-length-browser": "^4.2.2",
"@smithy/util-middleware": "^4.2.12",
"@smithy/util-stream": "^4.5.20",
"@smithy/util-utf8": "^4.2.2",
"@smithy/uuid": "^1.1.2",
"@aws-crypto/crc32": "5.2.0",
"@smithy/types": "^4.14.2",
"tslib": "^2.6.2"
},
"engines": {
@@ -3752,11 +3719,12 @@
}
},
"node_modules/@smithy/is-array-buffer": {
"version": "4.2.2",
"resolved": "https://registry.npmjs.org/@smithy/is-array-buffer/-/is-array-buffer-4.2.2.tgz",
"integrity": "sha512-n6rQ4N8Jj4YTQO3YFrlgZuwKodf4zUFs7EJIWH86pSCWBaAtAGBFfCM7Wx6D2bBJ2xqFNxGBSrUWswT3M0VJow==",
"version": "4.3.3",
"resolved": "https://registry.npmjs.org/@smithy/is-array-buffer/-/is-array-buffer-4.3.3.tgz",
"integrity": "sha512-RRxYqjUa/n8dRVkbhyuiRarppLzt4H/AtMUEFmiHlDy8o4wrgqAdzxsk9naemzu6iX67ZV375fNmX7Q8dynGKw==",
"license": "Apache-2.0",
"dependencies": {
"@smithy/core": "^3.24.3",
"tslib": "^2.6.2"
},
"engines": {
@@ -4020,9 +3988,9 @@
}
},
"node_modules/@smithy/types": {
"version": "4.14.1",
"resolved": "https://registry.npmjs.org/@smithy/types/-/types-4.14.1.tgz",
"integrity": "sha512-59b5HtSVrVR/eYNei3BUj3DCPKD/G7EtDDe7OEJE7i7FtQFugYo6MxbotS8mVJkLNVf8gYaAlEBwwtJ9HzhWSg==",
"version": "4.14.2",
"resolved": "https://registry.npmjs.org/@smithy/types/-/types-4.14.2.tgz",
"integrity": "sha512-P+otAxbV4CqBybp7EkcJCrig63yE2E7PuNVOmilVMRcx/O+QDzGULTrKsq4DV13gSfak9ObPrWaHl/9bL5YcWw==",
"license": "Apache-2.0",
"dependencies": {
"tslib": "^2.6.2"
@@ -4084,12 +4052,12 @@
}
},
"node_modules/@smithy/util-buffer-from": {
"version": "4.2.2",
"resolved": "https://registry.npmjs.org/@smithy/util-buffer-from/-/util-buffer-from-4.2.2.tgz",
"integrity": "sha512-FDXD7cvUoFWwN6vtQfEta540Y/YBe5JneK3SoZg9bThSoOAC/eGeYEua6RkBgKjGa/sz6Y+DuBZj3+YEY21y4Q==",
"version": "4.3.3",
"resolved": "https://registry.npmjs.org/@smithy/util-buffer-from/-/util-buffer-from-4.3.3.tgz",
"integrity": "sha512-5xlgilVaX96HdVlLZymKUa7vOTZtisOTxBJloM2J4PeRqyAWBeFIq0DnIxQISvwxT4rgJAvk7rHhB+GlCCKe8g==",
"license": "Apache-2.0",
"dependencies": {
"@smithy/is-array-buffer": "^4.2.2",
"@smithy/core": "^3.24.3",
"tslib": "^2.6.2"
},
"engines": {
@@ -4226,12 +4194,12 @@
}
},
"node_modules/@smithy/util-utf8": {
"version": "4.2.2",
"resolved": "https://registry.npmjs.org/@smithy/util-utf8/-/util-utf8-4.2.2.tgz",
"integrity": "sha512-75MeYpjdWRe8M5E3AW0O4Cx3UadweS+cwdXjwYGBW5h/gxxnbeZ877sLPX/ZJA9GVTlL/qG0dXP29JWFCD1Ayw==",
"version": "4.3.3",
"resolved": "https://registry.npmjs.org/@smithy/util-utf8/-/util-utf8-4.3.3.tgz",
"integrity": "sha512-c1QpRBn3aMsoqE64dd4Imgjy8Pynfw+eR7GkjElquxUFSnezwYVaOFm8JcYa+Bo/5ssbEyPKcT3+4bmrWYh6eQ==",
"license": "Apache-2.0",
"dependencies": {
"@smithy/util-buffer-from": "^4.2.2",
"@smithy/core": "^3.24.3",
"tslib": "^2.6.2"
},
"engines": {
@@ -4329,19 +4297,19 @@
"license": "MIT"
},
"node_modules/@trystero-p2p/core": {
"version": "0.23.0",
"resolved": "https://registry.npmjs.org/@trystero-p2p/core/-/core-0.23.0.tgz",
"integrity": "sha512-ozhtgxKDZH11Gdef0wH8xivwAE/L0/lDFvEcNFWPJWnHZlxWPPyfeonwE287ssGevQNi10vnj6x2ZcOi0n1bQQ==",
"version": "0.24.0",
"resolved": "https://registry.npmjs.org/@trystero-p2p/core/-/core-0.24.0.tgz",
"integrity": "sha512-W5ATiflgzZLE21fN2VA3YsK2yBJEzCvhmJ/9q2Vm3QT/gcdqDpcBxsO0DYCy/wE1PBEwoB+A75eBNtGIGAPdxw==",
"license": "MIT"
},
"node_modules/@trystero-p2p/nostr": {
"version": "0.23.0",
"resolved": "https://registry.npmjs.org/@trystero-p2p/nostr/-/nostr-0.23.0.tgz",
"integrity": "sha512-KSqUR2c1KVfv4zeErcntuegtyKzFTzNNiitIKGD0LiKA/4H3CeTF81ROk2h+X/PNvP4mv7Gp5eVxFYwfMu4Nrg==",
"version": "0.24.0",
"resolved": "https://registry.npmjs.org/@trystero-p2p/nostr/-/nostr-0.24.0.tgz",
"integrity": "sha512-LmsJSicsFU/rhmOWYaP/OxFl3rwGieX+q0eh0pAWUQM7IXbMu6tLC5+aAimtHitikPv9r6sck6EUTWMin8dBAw==",
"license": "MIT",
"dependencies": {
"@noble/secp256k1": "^3.0.0",
"@trystero-p2p/core": "0.23.0"
"@noble/secp256k1": "^3.1.0",
"@trystero-p2p/core": "0.24.0"
}
},
"node_modules/@tsconfig/svelte": {
@@ -4521,9 +4489,9 @@
"license": "MIT"
},
"node_modules/@types/node": {
"version": "24.12.2",
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.12.2.tgz",
"integrity": "sha512-A1sre26ke7HDIuY/M23nd9gfB+nrmhtYyMINbjI1zHJxYteKR6qSMX56FsmjMcDb3SMcjJg5BiRRgOCC/yBD0g==",
"version": "24.12.4",
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.12.4.tgz",
"integrity": "sha512-GUUEShf+PBCGW2KaXwcIt3Yk+e3pkKwWKb9GSyM9WQVE+ep2jzmHdGsHzu4wgcZy5fN9FBdVzjpBQsYlpfpgLA==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -5255,9 +5223,9 @@
"license": "MIT"
},
"node_modules/@wdio/config/node_modules/brace-expansion": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.3.tgz",
"integrity": "sha512-MCV/fYJEbqx68aE58kv2cA/kiky1G8vux3OR6/jbS+jIMe/6fJWa0DTzJU7dqijOWYwHi1t29FlfYI9uytqlpA==",
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.0.tgz",
"integrity": "sha512-TN1kCZAgdgweJhWWpgKYrQaMNHcDULHkWwQIspdtjV4Y5aurRdZpjAqn6yX3FPqTA9ngHCc4hJxMAMgGfve85w==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -5377,9 +5345,9 @@
}
},
"node_modules/@wdio/repl/node_modules/@types/node": {
"version": "20.19.39",
"resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.39.tgz",
"integrity": "sha512-orrrD74MBUyK8jOAD/r0+lfa1I2MO6I+vAkmAWzMYbCcgrN4lCrmK52gRFQq/JRxfYPfonkr4b0jcY7Olqdqbw==",
"version": "20.19.41",
"resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.41.tgz",
"integrity": "sha512-ECymXOukMnOoVkC2bb1Vc/w/836DXncOg5m8Xj1RH7xSHZJWNYY6Zh7EH477vcnD5egKNNfy2RpNOmuChhFPgQ==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -5407,9 +5375,9 @@
}
},
"node_modules/@wdio/types/node_modules/@types/node": {
"version": "20.19.39",
"resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.39.tgz",
"integrity": "sha512-orrrD74MBUyK8jOAD/r0+lfa1I2MO6I+vAkmAWzMYbCcgrN4lCrmK52gRFQq/JRxfYPfonkr4b0jcY7Olqdqbw==",
"version": "20.19.41",
"resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.41.tgz",
"integrity": "sha512-ECymXOukMnOoVkC2bb1Vc/w/836DXncOg5m8Xj1RH7xSHZJWNYY6Zh7EH477vcnD5egKNNfy2RpNOmuChhFPgQ==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -5532,9 +5500,9 @@
}
},
"node_modules/ajv": {
"version": "6.14.0",
"resolved": "https://registry.npmjs.org/ajv/-/ajv-6.14.0.tgz",
"integrity": "sha512-IWrosm/yrn43eiKqkfkHis7QioDleaXQHdDVPKg0FSwwd/DuvyX79TZnFOnYpB7dcsFAMmtFztZuXPDvSePkFw==",
"version": "6.15.0",
"resolved": "https://registry.npmjs.org/ajv/-/ajv-6.15.0.tgz",
"integrity": "sha512-fgFx7Hfoq60ytK2c7DhnF8jIvzYgOMxfugjLOSMHjLIPgenqa7S7oaagATUq99mV6IYvN2tRmC0wnTYX6iPbMw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -5623,9 +5591,9 @@
"license": "MIT"
},
"node_modules/archiver-utils/node_modules/brace-expansion": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.3.tgz",
"integrity": "sha512-MCV/fYJEbqx68aE58kv2cA/kiky1G8vux3OR6/jbS+jIMe/6fJWa0DTzJU7dqijOWYwHi1t29FlfYI9uytqlpA==",
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.0.tgz",
"integrity": "sha512-TN1kCZAgdgweJhWWpgKYrQaMNHcDULHkWwQIspdtjV4Y5aurRdZpjAqn6yX3FPqTA9ngHCc4hJxMAMgGfve85w==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -6234,9 +6202,9 @@
"license": "MIT"
},
"node_modules/brace-expansion": {
"version": "5.0.5",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.5.tgz",
"integrity": "sha512-VZznLgtwhn+Mact9tfiwx64fA9erHH/MCXEUfB/0bX/6Fz6ny5EGTXYltMocqg4xFAQZtnO3DHWWXi8RiuN7cQ==",
"version": "5.0.6",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
"integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
"license": "MIT",
"dependencies": {
"balanced-match": "^4.0.2"
@@ -7150,9 +7118,9 @@
}
},
"node_modules/dotenv": {
"version": "17.4.0",
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-17.4.0.tgz",
"integrity": "sha512-kCKF62fwtzwYm0IGBNjRUjtJgMfGapII+FslMHIjMR5KTnwEmBmWLDRSnc3XSNP8bNy34tekgQyDT0hr7pERRQ==",
"version": "17.4.2",
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-17.4.2.tgz",
"integrity": "sha512-nI4U3TottKAcAD9LLud4Cb7b2QztQMUEfHbvhTH09bqXTxnSie8WnjPALV/WMCrJZ6UV/qHJ6L03OqO3LcdYZw==",
"dev": true,
"license": "BSD-2-Clause",
"engines": {
@@ -7914,9 +7882,9 @@
"license": "MIT"
},
"node_modules/eslint-plugin-import/node_modules/brace-expansion": {
"version": "1.1.13",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.13.tgz",
"integrity": "sha512-9ZLprWS6EENmhEOpjCYW2c8VkmOvckIJZfkr7rBW6dObmfgJ/L1GpSYW5Hpo9lDz4D1+n0Ckz8rU7FwHDQiG/w==",
"version": "1.1.14",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.14.tgz",
"integrity": "sha512-MWPGfDxnyzKU7rNOW9SP/c50vi3xrmrua/+6hfPbCS2ABNWfx24vPidzvC7krjU/RTo235sV776ymlsMtGKj8g==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -8262,14 +8230,14 @@
}
},
"node_modules/eslint-plugin-react/node_modules/resolve": {
"version": "2.0.0-next.6",
"resolved": "https://registry.npmjs.org/resolve/-/resolve-2.0.0-next.6.tgz",
"integrity": "sha512-3JmVl5hMGtJ3kMmB3zi3DL25KfkCEyy3Tw7Gmw7z5w8M9WlwoPFnIvwChzu1+cF3iaK3sp18hhPz8ANeimdJfA==",
"version": "2.0.0-next.7",
"resolved": "https://registry.npmjs.org/resolve/-/resolve-2.0.0-next.7.tgz",
"integrity": "sha512-tqt+NBWwyaMgw3zDsnygx4CByWjQEJHOPMdslYhppaQSJUtL/D4JO9CcBBlhPoI8lz9oJIDXkwXfhF4aWqP8xQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"is-core-module": "^2.16.1",
"is-core-module": "^2.16.2",
"node-exports-info": "^1.6.0",
"object-keys": "^1.1.1",
"path-parse": "^1.0.7",
@@ -8432,19 +8400,6 @@
"url": "https://opencollective.com/eslint"
}
},
"node_modules/eslint/node_modules/@eslint/core": {
"version": "0.17.0",
"resolved": "https://registry.npmjs.org/@eslint/core/-/core-0.17.0.tgz",
"integrity": "sha512-yL/sLrpmtDaFEiUj1osRP4TI2MDz1AddJL+jZ7KSqvBuliN4xqYY54IfdN8qD8Toa6g1iloph1fxQNkjOxrrpQ==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"@types/json-schema": "^7.0.15"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
}
},
"node_modules/eslint/node_modules/balanced-match": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
@@ -8453,9 +8408,9 @@
"license": "MIT"
},
"node_modules/eslint/node_modules/brace-expansion": {
"version": "1.1.13",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.13.tgz",
"integrity": "sha512-9ZLprWS6EENmhEOpjCYW2c8VkmOvckIJZfkr7rBW6dObmfgJ/L1GpSYW5Hpo9lDz4D1+n0Ckz8rU7FwHDQiG/w==",
"version": "1.1.14",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.14.tgz",
"integrity": "sha512-MWPGfDxnyzKU7rNOW9SP/c50vi3xrmrua/+6hfPbCS2ABNWfx24vPidzvC7krjU/RTo235sV776ymlsMtGKj8g==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -9289,9 +9244,9 @@
"license": "MIT"
},
"node_modules/globby/node_modules/brace-expansion": {
"version": "1.1.13",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.13.tgz",
"integrity": "sha512-9ZLprWS6EENmhEOpjCYW2c8VkmOvckIJZfkr7rBW6dObmfgJ/L1GpSYW5Hpo9lDz4D1+n0Ckz8rU7FwHDQiG/w==",
"version": "1.1.14",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.14.tgz",
"integrity": "sha512-MWPGfDxnyzKU7rNOW9SP/c50vi3xrmrua/+6hfPbCS2ABNWfx24vPidzvC7krjU/RTo235sV776ymlsMtGKj8g==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -9443,9 +9398,9 @@
}
},
"node_modules/hasown": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.2.tgz",
"integrity": "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==",
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.3.tgz",
"integrity": "sha512-ej4AhfhfL2Q2zpMmLo7U1Uv9+PyhIZpgQLGT1F9miIGmiCJIoCgSmczFdrc97mWT4kVY72KA+WnnhJ5pghSvSg==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -9786,13 +9741,13 @@
}
},
"node_modules/is-core-module": {
"version": "2.16.1",
"resolved": "https://registry.npmjs.org/is-core-module/-/is-core-module-2.16.1.tgz",
"integrity": "sha512-UfoeMA6fIJ8wTYFEUjelnaGI67v6+N7qXJEvQuIGa99l4xsCruSYOVSQ0uPANn4dAzm8lkYPaKLrrijLq7x23w==",
"version": "2.16.2",
"resolved": "https://registry.npmjs.org/is-core-module/-/is-core-module-2.16.2.tgz",
"integrity": "sha512-evOr8xfXKxE6qSR0hSXL2r3sd7ALj8+7jQEUvPYcm5sgZFdJ+AYzT6yNmJenvIYQBgIGwfwz08sL8zoL7yq2BA==",
"dev": true,
"license": "MIT",
"dependencies": {
"hasown": "^2.0.2"
"hasown": "^2.0.3"
},
"engines": {
"node": ">= 0.4"
@@ -10973,9 +10928,9 @@
"license": "MIT"
},
"node_modules/lru-cache": {
"version": "11.2.7",
"resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.2.7.tgz",
"integrity": "sha512-aY/R+aEsRelme17KGQa/1ZSIpLpNYYrhcrepKTZgE+W3WM16YMCaPwOHLHsmopZHELU0Ojin1lPVxKR0MihncA==",
"version": "11.4.0",
"resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.4.0.tgz",
"integrity": "sha512-W+R+kFL4HgVxONq2bhXPi3bGpzGe/yEhVOp233qw9wCRtgncJ15P3bC+e4zZMu4Cq7d+WAJjXGW0uUkifhcatA==",
"dev": true,
"license": "BlueOak-1.0.0",
"engines": {
@@ -12695,9 +12650,9 @@
"license": "MIT"
},
"node_modules/readdir-glob/node_modules/brace-expansion": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.3.tgz",
"integrity": "sha512-MCV/fYJEbqx68aE58kv2cA/kiky1G8vux3OR6/jbS+jIMe/6fJWa0DTzJU7dqijOWYwHi1t29FlfYI9uytqlpA==",
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.0.tgz",
"integrity": "sha512-TN1kCZAgdgweJhWWpgKYrQaMNHcDULHkWwQIspdtjV4Y5aurRdZpjAqn6yX3FPqTA9ngHCc4hJxMAMgGfve85w==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -12817,12 +12772,13 @@
"license": "MIT"
},
"node_modules/resolve": {
"version": "1.22.11",
"resolved": "https://registry.npmjs.org/resolve/-/resolve-1.22.11.tgz",
"integrity": "sha512-RfqAvLnMl313r7c9oclB1HhUEAezcpLjz95wFH4LVuhk9JF/r22qmVP9AMmOU4vMX7Q8pN8jwNg/CSpdFnMjTQ==",
"version": "1.22.12",
"resolved": "https://registry.npmjs.org/resolve/-/resolve-1.22.12.tgz",
"integrity": "sha512-TyeJ1zif53BPfHootBGwPRYT1RUt6oGWsaQr8UyZW/eAm9bKoijtvruSDEmZHm92CwS9nj7/fWttqPCgzep8CA==",
"dev": true,
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"is-core-module": "^2.16.1",
"path-parse": "^1.0.7",
"supports-preserve-symlinks-flag": "^1.0.0"
@@ -13119,9 +13075,9 @@
"license": "MIT"
},
"node_modules/semver": {
"version": "7.7.4",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.7.4.tgz",
"integrity": "sha512-vFKC2IEtQnVhpT78h1Yp8wzwrf8CM+MzKMHGJZfBtzhZNycRFnXsHk6E5TxIkkMsgNS7mdX3AGB7x2QM2di4lA==",
"version": "7.8.0",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.8.0.tgz",
"integrity": "sha512-AcM7dV/5ul4EekoQ29Agm5vri8JNqRyj39o0qpX6vDF2GZrtutZl5RwgD1XnZjiTAfncsJhMI48QQH3sN87YNA==",
"dev": true,
"license": "ISC",
"bin": {
@@ -15293,9 +15249,9 @@
}
},
"node_modules/undici": {
"version": "7.24.7",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.24.7.tgz",
"integrity": "sha512-H/nlJ/h0ggGC+uRL3ovD+G0i4bqhvsDOpbDv7At5eFLlj2b41L8QliGbnl2H7SnDiYhENphh1tQFJZf+MyfLsQ==",
"version": "7.25.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.25.0.tgz",
"integrity": "sha512-xXnp4kTyor2Zq+J1FfPI6Eq3ew5h6Vl0F/8d9XU5zZQf1tX9s2Su1/3PiMmUANFULpmksxkClamIZcaUqryHsQ==",
"dev": true,
"license": "MIT",
"engines": {
@@ -16157,9 +16113,9 @@
}
},
"node_modules/vitest/node_modules/es-module-lexer": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.0.0.tgz",
"integrity": "sha512-5POEcUuZybH7IdmGsD8wlf0AI55wMecM9rVBTI/qEAy2c1kTOm3DjFYjrBdI2K3BaJjJYfYFeRtM0t9ssnRuxw==",
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.1.0.tgz",
"integrity": "sha512-n27zTYMjYu1aj4MjCWzSP7G9r75utsaoc8m61weK+W8JMBGGQybd43GstCXZ3WNmSFtGT9wi59qQTW6mhTR5LQ==",
"dev": true,
"license": "MIT"
},
@@ -16240,9 +16196,9 @@
}
},
"node_modules/webdriver/node_modules/@types/node": {
"version": "20.19.39",
"resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.39.tgz",
"integrity": "sha512-orrrD74MBUyK8jOAD/r0+lfa1I2MO6I+vAkmAWzMYbCcgrN4lCrmK52gRFQq/JRxfYPfonkr4b0jcY7Olqdqbw==",
"version": "20.19.41",
"resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.41.tgz",
"integrity": "sha512-ECymXOukMnOoVkC2bb1Vc/w/836DXncOg5m8Xj1RH7xSHZJWNYY6Zh7EH477vcnD5egKNNfy2RpNOmuChhFPgQ==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -16250,9 +16206,9 @@
}
},
"node_modules/webdriver/node_modules/undici": {
"version": "6.24.1",
"resolved": "https://registry.npmjs.org/undici/-/undici-6.24.1.tgz",
"integrity": "sha512-sC+b0tB1whOCzbtlx20fx3WgCXwkW627p4EA9uM+/tNNPkSS+eSEld6pAs9nDv7WbY1UUljBMYPtu9BCOrCWKA==",
"version": "6.25.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-6.25.0.tgz",
"integrity": "sha512-ZgpWDC5gmNiuY9CnLVXEH8rl50xhRCuLNA97fAUnKi8RRuV4E6KG31pDTsLVUKnohJE0I3XDrTeEydAXRw47xg==",
"dev": true,
"license": "MIT",
"engines": {
@@ -16312,9 +16268,9 @@
}
},
"node_modules/webdriverio/node_modules/@types/node": {
"version": "20.19.39",
"resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.39.tgz",
"integrity": "sha512-orrrD74MBUyK8jOAD/r0+lfa1I2MO6I+vAkmAWzMYbCcgrN4lCrmK52gRFQq/JRxfYPfonkr4b0jcY7Olqdqbw==",
"version": "20.19.41",
"resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.41.tgz",
"integrity": "sha512-ECymXOukMnOoVkC2bb1Vc/w/836DXncOg5m8Xj1RH7xSHZJWNYY6Zh7EH477vcnD5egKNNfy2RpNOmuChhFPgQ==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -16335,9 +16291,9 @@
"license": "BSD-2-Clause"
},
"node_modules/werift": {
"version": "0.22.9",
"resolved": "https://registry.npmjs.org/werift/-/werift-0.22.9.tgz",
"integrity": "sha512-TE9AxskSRWBMYm0MBRllfnKVXQelqC76JCvyolQyVWpmKabfY5BA/cuvkGg+71JWn3QrGih1YWtpIWGPqoxcoA==",
"version": "0.23.0",
"resolved": "https://registry.npmjs.org/werift/-/werift-0.23.0.tgz",
"integrity": "sha512-/WcIN5DHFG9Ri4anGOmIkp8gxBGFMWSIB/m4sfZ5CWlLfD3iMhiaAUuTBuc+KV3SY9NDmvmLtiN2uaM7k3lVzw==",
"license": "MIT",
"dependencies": {
"@fidm/x509": "^1.2.1",
@@ -16778,9 +16734,9 @@
"license": "BSD"
},
"node_modules/ws": {
"version": "8.20.0",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.20.0.tgz",
"integrity": "sha512-sAt8BhgNbzCtgGbt2OxmpuryO63ZoDk/sqaB/znQm94T4fCEsy/yV+7CdC1kJhOU9lboAEU7R3kquuycDoibVA==",
"version": "8.20.1",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.20.1.tgz",
"integrity": "sha512-It4dO0K5v//JtTXuPkfEOaI3uUN87iYPnqo/ZzqCoG3g8uhA66QUMs/SrM0YK7/NAu+r4LMh/9dq2A7k+rHs+w==",
"dev": true,
"license": "MIT",
"engines": {
@@ -16848,9 +16804,9 @@
"license": "ISC"
},
"node_modules/yaml": {
"version": "2.8.3",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.8.3.tgz",
"integrity": "sha512-AvbaCLOO2Otw/lW5bmh9d/WEdcDFdQp2Z2ZUH3pX9U2ihyUY0nvLv7J6TrWowklRGPYbB/IuIMfYgxaCPg5Bpg==",
"version": "2.9.0",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.0.tgz",
"integrity": "sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==",
"dev": true,
"license": "ISC",
"bin": {

View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.25.64",
"version": "0.25.68",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -130,7 +130,7 @@
"@smithy/middleware-apply-body-checksum": "^4.3.9",
"@smithy/protocol-http": "^5.3.9",
"@smithy/querystring-builder": "^4.2.9",
"@trystero-p2p/nostr": "^0.23.0",
"@trystero-p2p/nostr": "^0.24.0",
"chokidar": "^4.0.0",
"commander": "^14.0.3",
"obsidian": "^1.12.3",
@@ -143,7 +143,7 @@
"octagonal-wheels": "^0.1.45",
"pouchdb-adapter-leveldb": "^9.0.0",
"qrcode-generator": "^1.4.4",
"werift": "^0.22.9",
"werift": "^0.23.0",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
}
}

View File

@@ -5,4 +5,5 @@ test/test-init.local.sh
node_modules
.*.json
*.env
!.test.env
!.test.env
bench-results

View File

@@ -74,6 +74,12 @@ livesync-cli [database-path] [command] [args...]
- `pull <src> <dst>`: Pull a file `<src>` from the database into local file `<dst>`.
- `cat <src>`: Read a file from the database and write to stdout.
- `put <dst>`: Read from stdin and write to the database path `<dst>`.
- `remote-add <name> <connstr>`: Add a remote configuration from a connection string.
- `remote-rm <remote-id>`: Remove a remote configuration by ID.
- `remote-ls`: List remote configurations (`id`, `name`, `active/inactive`, redacted URI).
- `remote-export <remote-id>`: Export the stored connection string by remote ID.
- `remote-set <remote-id> <connstr>`: Replace the stored connection string by remote ID.
- `remote-activate <remote-id>`: Activate a remote configuration by ID.
- `init-settings [file]`: Create a default settings file.
### Examples
@@ -252,6 +258,14 @@ livesync-cli /path/to/your-local-database --settings /path/to/settings.json rm /
# Resolve conflict by keeping a specific revision
livesync-cli /path/to/your-local-database --settings /path/to/settings.json resolve /vault/path/file.md 3-abcdef
# Add/list/activate/remove remote configurations
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-add main "sls+https://user:pass@example.com/db"
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-ls
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-export remote-abc123
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-set remote-abc123 "sls+p2p://room-abc?passphrase=secret"
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-activate remote-abc123
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-rm remote-abc123
```
### Configuration

View File

@@ -3,6 +3,8 @@ import * as path from "path";
import { decodeSettingsFromSetupURI } from "@lib/API/processSetting";
import { configURIBase } from "@lib/common/models/shared.const";
import { DEFAULT_SETTINGS, type FilePathWithPrefix, type ObsidianLiveSyncSettings } from "@lib/common/types";
import { ConnectionStringParser } from "@lib/common/ConnectionString";
import { activateRemoteConfiguration, createRemoteConfigurationId } from "@lib/serviceFeatures/remoteConfig";
import { stripAllPrefixes } from "@lib/string_and_binary/path";
import type { CLICommandContext, CLIOptions } from "./types";
import { promptForPassphrase, readStdinAsUtf8, toArrayBuffer, toDatabaseRelativePath } from "./utils";
@@ -10,6 +12,10 @@ import { collectPeers, openP2PHost, parseTimeoutSeconds, syncWithPeer } from "./
import { performFullScan } from "@lib/serviceFeatures/offlineScanner";
import { UnresolvedErrorManager } from "@lib/services/base/UnresolvedErrorManager";
function redactConnectionString(uri: string): string {
return uri.replace(/\/\/([^@/]+)@/u, "//***@");
}
export async function runCommand(options: CLIOptions, context: CLICommandContext): Promise<boolean> {
const { databasePath, core, settingsPath } = context;
@@ -469,5 +475,206 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
return await performFullScan(core as any, log, errorManager, false, true);
}
if (options.command === "remote-add") {
if (options.commandArgs.length < 2) {
throw new Error("remote-add requires two arguments: <name> <connstr>");
}
const name = options.commandArgs[0].trim();
const connectionString = options.commandArgs[1].trim();
if (!name) {
throw new Error("remote-add requires a non-empty name");
}
if (!connectionString) {
throw new Error("remote-add requires a non-empty connection string");
}
const parsed = ConnectionStringParser.parse(connectionString);
const canonicalUri = ConnectionStringParser.serialize(parsed);
const id = createRemoteConfigurationId();
let activated = false;
await core.services.setting.updateSettings((currentSettings) => {
currentSettings.remoteConfigurations ||= {};
currentSettings.remoteConfigurations[id] = {
id,
name,
uri: canonicalUri,
isEncrypted: false,
};
if (!currentSettings.activeConfigurationId) {
currentSettings.activeConfigurationId = id;
const applied = activateRemoteConfiguration(currentSettings, id);
activated = applied !== false;
}
return currentSettings;
}, true);
if (activated) {
await core.services.control.applySettings();
}
process.stdout.write(`${id}\t${name}\t${redactConnectionString(canonicalUri)}\n`);
return true;
}
if (options.command === "remote-rm") {
if (options.commandArgs.length < 1) {
throw new Error("remote-rm requires one argument: <remote-id>");
}
const id = options.commandArgs[0].trim();
if (!id) {
throw new Error("remote-rm requires a non-empty remote-id");
}
const current = core.services.setting.currentSettings();
if (!current.remoteConfigurations?.[id]) {
process.stderr.write(`[Info] Remote configuration not found: ${id}\n`);
return false;
}
let switchedActive = false;
await core.services.setting.updateSettings((currentSettings) => {
const configs = currentSettings.remoteConfigurations || {};
delete configs[id];
currentSettings.remoteConfigurations = configs;
if (currentSettings.activeConfigurationId === id) {
const nextActiveId = Object.keys(configs)[0] || "";
currentSettings.activeConfigurationId = nextActiveId;
switchedActive = nextActiveId !== "";
if (nextActiveId !== "") {
activateRemoteConfiguration(currentSettings, nextActiveId);
}
}
if (currentSettings.P2P_ActiveRemoteConfigurationId === id) {
currentSettings.P2P_ActiveRemoteConfigurationId = "";
}
return currentSettings;
}, true);
if (switchedActive) {
await core.services.control.applySettings();
}
console.error(`[Command] remote-rm ${id}`);
return true;
}
if (options.command === "remote-ls") {
const settings = core.services.setting.currentSettings();
const configs = Object.values(settings.remoteConfigurations || {});
configs.sort((a, b) => a.name.localeCompare(b.name));
if (configs.length === 0) {
process.stderr.write("[Info] No remote configurations found.\n");
return true;
}
const lines = configs.map((config) => {
const status = config.id === settings.activeConfigurationId ? "active" : "inactive";
return `${config.id}\t${config.name}\t${status}\t${redactConnectionString(config.uri)}`;
});
process.stdout.write(lines.join("\n") + "\n");
return true;
}
if (options.command === "remote-export") {
if (options.commandArgs.length < 1) {
throw new Error("remote-export requires one argument: <remote-id>");
}
const id = options.commandArgs[0].trim();
if (!id) {
throw new Error("remote-export requires a non-empty remote-id");
}
const config = core.services.setting.currentSettings().remoteConfigurations?.[id];
if (!config) {
process.stderr.write(`[Info] Remote configuration not found: ${id}\n`);
return false;
}
process.stdout.write(`${config.uri}\n`);
return true;
}
if (options.command === "remote-set") {
if (options.commandArgs.length < 2) {
throw new Error("remote-set requires two arguments: <remote-id> <connstr>");
}
const id = options.commandArgs[0].trim();
const connectionString = options.commandArgs[1].trim();
if (!id) {
throw new Error("remote-set requires a non-empty remote-id");
}
if (!connectionString) {
throw new Error("remote-set requires a non-empty connection string");
}
const parsed = ConnectionStringParser.parse(connectionString);
const canonicalUri = ConnectionStringParser.serialize(parsed);
let switchedActive = false;
await core.services.setting.updateSettings((currentSettings) => {
const config = currentSettings.remoteConfigurations?.[id];
if (!config) {
return currentSettings;
}
config.uri = canonicalUri;
if (currentSettings.activeConfigurationId === id) {
const activated = activateRemoteConfiguration(currentSettings, id);
switchedActive = activated !== false;
if (activated) {
return activated;
}
}
return currentSettings;
}, true);
const updated = core.services.setting.currentSettings().remoteConfigurations?.[id];
if (!updated) {
process.stderr.write(`[Info] Remote configuration not found: ${id}\n`);
return false;
}
if (switchedActive) {
await core.services.control.applySettings();
}
console.error(`[Command] remote-set ${id}`);
return true;
}
if (options.command === "remote-activate") {
if (options.commandArgs.length < 1) {
throw new Error("remote-activate requires one argument: <remote-id>");
}
const id = options.commandArgs[0].trim();
if (!id) {
throw new Error("remote-activate requires a non-empty remote-id");
}
let switched = false;
await core.services.setting.updateSettings((currentSettings) => {
const activated = activateRemoteConfiguration(currentSettings, id);
if (activated) {
switched = true;
return activated;
}
return currentSettings;
}, true);
if (!switched) {
process.stderr.write(`[Info] Failed to activate remote configuration: ${id}\n`);
return false;
}
await core.services.control.applySettings();
console.error(`[Command] remote-activate ${id}`);
return true;
}
throw new Error(`Unsupported command: ${options.command}`);
}

View File

@@ -1,12 +1,19 @@
import * as processSetting from "@lib/API/processSetting";
import { ConnectionStringParser } from "@lib/common/ConnectionString";
import { configURIBase } from "@lib/common/models/shared.const";
import { DEFAULT_SETTINGS } from "@lib/common/types";
import { DEFAULT_SETTINGS, REMOTE_COUCHDB, REMOTE_MINIO, REMOTE_P2P } from "@lib/common/types";
import { describe, expect, it, vi, beforeEach, afterEach } from "vitest";
import { runCommand } from "./runCommand";
import type { CLIOptions } from "./types";
import * as commandUtils from "./utils";
function createCoreMock() {
const liveSettings = {
...DEFAULT_SETTINGS,
remoteConfigurations: {},
activeConfigurationId: "",
P2P_ActiveRemoteConfigurationId: "",
} as any;
return {
services: {
control: {
@@ -16,6 +23,10 @@ function createCoreMock() {
setting: {
applyExternalSettings: vi.fn(async () => {}),
applyPartial: vi.fn(async () => {}),
currentSettings: vi.fn(() => liveSettings),
updateSettings: vi.fn(async (updater: any) => {
updater(liveSettings);
}),
},
},
serviceModules: {
@@ -56,6 +67,115 @@ async function createSetupURI(passphrase: string): Promise<string> {
return await processSetting.encodeSettingsToSetupURI(settings, passphrase);
}
function captureStdout() {
const writes: string[] = [];
const spy = vi.spyOn(process.stdout, "write").mockImplementation((chunk: any) => {
writes.push(typeof chunk === "string" ? chunk : String(chunk));
return true;
});
return {
spy,
lines: () =>
writes
.join("")
.split("\n")
.map((e) => e.trim())
.filter((e) => e.length > 0),
};
}
function parseAddedRemoteIdFromLines(lines: string[]): string {
// remote-add prints: <id>\t<name>\t<redacted-connstr>
const last = lines.length > 0 ? lines[lines.length - 1] : "";
return last.split("\t")[0] || "";
}
type ProtocolFixture = {
protocol: string;
connectionString: string;
assertProjectedFields: (settings: any) => void;
};
const protocolFixtures: ProtocolFixture[] = [
{
protocol: "couchdb",
connectionString: ConnectionStringParser.serialize({
type: "couchdb",
settings: {
couchDB_URI: "https://db.example.com:5984",
couchDB_USER: "user1",
couchDB_PASSWORD: "pass1",
couchDB_DBNAME: "vault1",
couchDB_CustomHeaders: "",
useJWT: false,
jwtAlgorithm: "",
jwtKey: "",
jwtKid: "",
jwtSub: "",
jwtExpDuration: 5,
useRequestAPI: false,
},
}),
assertProjectedFields: (settings) => {
expect(settings.remoteType).toBe(REMOTE_COUCHDB);
expect(settings.couchDB_URI).toBe("https://db.example.com:5984");
expect(settings.couchDB_USER).toBe("user1");
expect(settings.couchDB_PASSWORD).toBe("pass1");
expect(settings.couchDB_DBNAME).toBe("vault1");
},
},
{
protocol: "s3",
connectionString: ConnectionStringParser.serialize({
type: "s3",
settings: {
accessKey: "ak",
secretKey: "sk",
endpoint: "https://s3.example.com",
bucket: "bucket-1",
region: "ap-northeast-1",
bucketPrefix: "vault/",
useCustomRequestHandler: true,
bucketCustomHeaders: "x-test:1",
forcePathStyle: false,
},
}),
assertProjectedFields: (settings) => {
expect(settings.remoteType).toBe(REMOTE_MINIO);
expect(settings.accessKey).toBe("ak");
expect(settings.secretKey).toBe("sk");
expect(settings.endpoint).toBe("https://s3.example.com");
expect(settings.bucket).toBe("bucket-1");
expect(settings.region).toBe("ap-northeast-1");
},
},
{
protocol: "p2p",
connectionString: ConnectionStringParser.serialize({
type: "p2p",
settings: {
P2P_Enabled: false,
P2P_roomID: "room-abc",
P2P_passphrase: "pass-123",
P2P_relays: "wss://relay.example",
P2P_AppID: "self-hosted-livesync",
P2P_AutoStart: true,
P2P_AutoBroadcast: false,
P2P_turnServers: "turn:turn.example:3478",
P2P_turnUsername: "turn-user",
P2P_turnCredential: "turn-pass",
},
}),
assertProjectedFields: (settings) => {
expect(settings.remoteType).toBe(REMOTE_P2P);
expect(settings.P2P_roomID).toBe("room-abc");
expect(settings.P2P_passphrase).toBe("pass-123");
expect(settings.P2P_relays).toBe("wss://relay.example");
expect(settings.P2P_AppID).toBe("self-hosted-livesync");
},
},
];
describe("runCommand abnormal cases", () => {
const context = {
databasePath: "/tmp/vault",
@@ -202,4 +322,254 @@ describe("runCommand abnormal cases", () => {
expect(core.services.setting.applyExternalSettings).not.toHaveBeenCalled();
expect(core.services.control.applySettings).not.toHaveBeenCalled();
});
it("remote-add stores canonical URI and prints the created id", async () => {
const core = createCoreMock();
const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true);
const result = await runCommand(makeOptions("remote-add", ["my-remote", "sls+https://example.com/db"]), {
...context,
core,
});
expect(result).toBe(true);
const settings = core.services.setting.currentSettings();
const ids = Object.keys(settings.remoteConfigurations);
expect(ids.length).toBe(1);
expect(settings.remoteConfigurations[ids[0]].name).toBe("my-remote");
expect(settings.remoteConfigurations[ids[0]].uri).toContain("sls+https://example.com/db");
expect(settings.activeConfigurationId).toBe(ids[0]);
expect(stdout).toHaveBeenCalled();
});
it("remote-activate switches active remote and applies settings", async () => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://example.com/db1",
isEncrypted: false,
};
settings.remoteConfigurations.r2 = {
id: "r2",
name: "R2",
uri: "sls+https://example.com/db2",
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-activate", ["r2"]), {
...context,
core,
});
expect(result).toBe(true);
expect(settings.activeConfigurationId).toBe("r2");
expect(core.services.control.applySettings).toHaveBeenCalledTimes(1);
});
it("remote-rm removes active remote and promotes first remaining", async () => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://example.com/db1",
isEncrypted: false,
};
settings.remoteConfigurations.r2 = {
id: "r2",
name: "R2",
uri: "sls+https://example.com/db2",
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-rm", ["r1"]), {
...context,
core,
});
expect(result).toBe(true);
expect(settings.remoteConfigurations.r1).toBeUndefined();
expect(settings.activeConfigurationId).toBe("r2");
expect(core.services.control.applySettings).toHaveBeenCalledTimes(1);
});
it("remote-export prints the exact stored connection string", async () => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://example.com/db?db=vault",
isEncrypted: false,
};
const stdout = captureStdout();
const result = await runCommand(makeOptions("remote-export", ["r1"]), {
...context,
core,
});
expect(result).toBe(true);
const outLines = stdout.lines();
expect(outLines.length > 0 ? outLines[outLines.length - 1] : "").toBe("sls+https://example.com/db?db=vault");
expect(stdout.spy).toHaveBeenCalled();
});
it("remote-set updates URI and applies settings when target is active", async () => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://old.example/db",
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-set", ["r1", "sls+https://new.example/db"]), {
...context,
core,
});
expect(result).toBe(true);
expect(settings.remoteConfigurations.r1.uri).toContain("sls+https://new.example/db");
expect(core.services.control.applySettings).toHaveBeenCalledTimes(1);
});
it.each(protocolFixtures)(
"remote-activate projects effective settings for $protocol",
async ({ connectionString, assertProjectedFields }) => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://old.example/?db=old",
isEncrypted: false,
};
settings.remoteConfigurations.r2 = {
id: "r2",
name: "R2",
uri: connectionString,
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-activate", ["r2"]), {
...context,
core,
});
expect(result).toBe(true);
expect(settings.activeConfigurationId).toBe("r2");
assertProjectedFields(settings);
}
);
it.each(protocolFixtures)(
"remote-set projects effective settings for active remote ($protocol)",
async ({ connectionString, assertProjectedFields }) => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://old.example/?db=old",
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-set", ["r1", connectionString]), {
...context,
core,
});
expect(result).toBe(true);
assertProjectedFields(settings);
}
);
it.each(protocolFixtures)(
"remote-rm projects promoted active remote effective settings for $protocol",
async ({ connectionString, assertProjectedFields }) => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://old.example/?db=old",
isEncrypted: false,
};
settings.remoteConfigurations.r2 = {
id: "r2",
name: "R2",
uri: connectionString,
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-rm", ["r1"]), {
...context,
core,
});
expect(result).toBe(true);
expect(settings.activeConfigurationId).toBe("r2");
assertProjectedFields(settings);
}
);
it.each([
["couchdb", "sls+https://user:pass@example.com:5984/?db=vault"] as const,
[
"s3",
"sls+s3://ak:sk@example.com/?endpoint=https%3A%2F%2Fs3.example.com&bucket=my-bucket&region=ap-northeast-1",
] as const,
[
"p2p",
"sls+p2p://room-abc?passphrase=pass-123&relays=wss%3A%2F%2Frelay.example&appId=self-hosted-livesync",
] as const,
])("remote command round-trip works for %s", async (_protocol, initialConnStr) => {
const core = createCoreMock();
const addOut = captureStdout();
const addResult = await runCommand(makeOptions("remote-add", ["rt", initialConnStr]), {
...context,
core,
});
expect(addResult).toBe(true);
const remoteId = parseAddedRemoteIdFromLines(addOut.lines());
expect(remoteId).not.toBe("");
const export1Out = captureStdout();
const export1Result = await runCommand(makeOptions("remote-export", [remoteId]), {
...context,
core,
});
expect(export1Result).toBe(true);
const export1Lines = export1Out.lines();
const exported1 = export1Lines.length > 0 ? export1Lines[export1Lines.length - 1] : "";
expect(exported1).toBe(ConnectionStringParser.serialize(ConnectionStringParser.parse(initialConnStr)));
const roundTripInput = ConnectionStringParser.serialize(ConnectionStringParser.parse(exported1));
const setResult = await runCommand(makeOptions("remote-set", [remoteId, roundTripInput]), {
...context,
core,
});
expect(setResult).toBe(true);
const export2Out = captureStdout();
const export2Result = await runCommand(makeOptions("remote-export", [remoteId]), {
...context,
core,
});
expect(export2Result).toBe(true);
const export2Lines = export2Out.lines();
const exported2 = export2Lines.length > 0 ? export2Lines[export2Lines.length - 1] : "";
expect(exported2).toBe(roundTripInput);
});
});

View File

@@ -20,6 +20,12 @@ export type CLICommand =
| "rm"
| "resolve"
| "mirror"
| "remote-add"
| "remote-rm"
| "remote-ls"
| "remote-export"
| "remote-set"
| "remote-activate"
| "init-settings";
export interface CLIOptions {
@@ -67,5 +73,11 @@ export const VALID_COMMANDS = new Set([
"rm",
"resolve",
"mirror",
"remote-add",
"remote-rm",
"remote-ls",
"remote-export",
"remote-set",
"remote-activate",
"init-settings",
] as const);

View File

@@ -62,6 +62,16 @@ Commands:
rm <path> Mark a file as deleted in local database
resolve <path> <rev> Resolve conflicts by keeping <rev> and deleting others
mirror [vault-path] Mirror database contents to the local file system (vault-path defaults to database-path)
remote-add <name> <connstr>
Add a remote configuration from a connection string
remote-rm <remote-id> Remove a remote configuration by ID
remote-ls List stored remote configurations
remote-export <remote-id>
Export a remote connection string by ID
remote-set <remote-id> <connstr>
Replace a stored remote connection string by ID
remote-activate <remote-id>
Activate a stored remote configuration by ID
Options:
--interval <N>, -i <N> (daemon only) Poll CouchDB every N seconds instead of using the _changes feed
@@ -84,6 +94,12 @@ Examples:
livesync-cli ./my-database info notes/hello.md
livesync-cli ./my-database rm notes/hello.md
livesync-cli ./my-database resolve notes/hello.md 3-abcdef
livesync-cli ./my-database remote-add my-remote "sls+https://user:pass@example.com/db"
livesync-cli ./my-database remote-ls
livesync-cli ./my-database remote-export remote-abc123
livesync-cli ./my-database remote-set remote-abc123 "sls+s3://ak:sk@example.com/?endpoint=https%3A%2F%2Fs3.example.com&bucket=mybucket"
livesync-cli ./my-database remote-activate remote-abc123
livesync-cli ./my-database remote-rm remote-abc123
livesync-cli init-settings ./data.json
livesync-cli ./my-database --verbose
`);
@@ -229,6 +245,9 @@ export async function main() {
options.command === "cat" ||
options.command === "cat-rev" ||
options.command === "ls" ||
options.command === "remote-add" ||
options.command === "remote-ls" ||
options.command === "remote-export" ||
options.command === "p2p-peers" ||
options.command === "info" ||
options.command === "rm" ||

View File

@@ -86,6 +86,56 @@ describe("CLI parseArgs", () => {
expect(parsed.commandArgs).toEqual([]);
});
it("parses remote-add command", () => {
process.argv = [
"node",
"livesync-cli",
"./databasePath",
"remote-add",
"my-remote",
"sls+https://user:pass@example.com/db",
];
const parsed = parseArgs();
expect(parsed.databasePath).toBe("./databasePath");
expect(parsed.command).toBe("remote-add");
expect(parsed.commandArgs).toEqual(["my-remote", "sls+https://user:pass@example.com/db"]);
});
it("parses remote-activate command", () => {
process.argv = ["node", "livesync-cli", "./databasePath", "remote-activate", "remote-abc"];
const parsed = parseArgs();
expect(parsed.databasePath).toBe("./databasePath");
expect(parsed.command).toBe("remote-activate");
expect(parsed.commandArgs).toEqual(["remote-abc"]);
});
it("parses remote-export command", () => {
process.argv = ["node", "livesync-cli", "./databasePath", "remote-export", "remote-abc"];
const parsed = parseArgs();
expect(parsed.databasePath).toBe("./databasePath");
expect(parsed.command).toBe("remote-export");
expect(parsed.commandArgs).toEqual(["remote-abc"]);
});
it("parses remote-set command", () => {
process.argv = [
"node",
"livesync-cli",
"./databasePath",
"remote-set",
"remote-abc",
"sls+p2p://room-1?passphrase=abc",
];
const parsed = parseArgs();
expect(parsed.databasePath).toBe("./databasePath");
expect(parsed.command).toBe("remote-set");
expect(parsed.commandArgs).toEqual(["remote-abc", "sls+p2p://room-1?passphrase=abc"]);
});
it("parses --interval flag with valid integer", () => {
process.argv = ["node", "livesync-cli", "./vault", "--interval", "30"];
const parsed = parseArgs();

View File

@@ -0,0 +1,312 @@
import { TempDir } from "./helpers/temp.ts";
import { applyRemoteSyncSettings, initSettingsFile } from "./helpers/settings.ts";
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
import { startCouchdb, stopCouchdb } from "./helpers/docker.ts";
import { createDeterministicDataset, type DatasetEntry } from "./helpers/dataset.ts";
type BenchmarkConfig = {
couchdbBackendUri: string;
couchdbProxyUri: string;
couchdbUser: string;
couchdbPassword: string;
couchdbDbname: string;
datasetDirName: string;
datasetSeed: string;
mdFileCount: number;
mdMinSizeBytes: number;
mdMaxSizeBytes: number;
binFileCount: number;
binSizeBytes: number;
syncTimeoutSeconds: number;
requestedRttMs: number;
passphrase: string;
encrypt: boolean;
};
function readEnvString(name: string, fallback: string): string {
const value = Deno.env.get(name)?.trim();
return value && value.length > 0 ? value : fallback;
}
function readEnvNumber(name: string, fallback: number): number {
const raw = Deno.env.get(name);
if (raw === undefined || raw.trim() === "") {
return fallback;
}
const parsed = Number(raw);
if (!Number.isFinite(parsed) || parsed <= 0) {
throw new Error(`${name} must be a positive number, got '${raw}'`);
}
return parsed;
}
function readEnvBool(name: string, fallback: boolean): boolean {
const raw = Deno.env.get(name);
if (raw === undefined || raw.trim() === "") {
return fallback;
}
return /^(1|true|yes|on)$/i.test(raw.trim());
}
function nowMs(): number {
return performance.now();
}
function formatMs(value: number): string {
return `${value.toFixed(1)} ms`;
}
function formatBytes(value: number): string {
if (value < 1024) {
return `${value} B`;
}
const kib = value / 1024;
if (kib < 1024) {
return `${kib.toFixed(1)} KiB`;
}
return `${(kib / 1024).toFixed(1)} MiB`;
}
function buildConfig(): BenchmarkConfig {
return {
couchdbBackendUri: readEnvString("BENCH_COUCHDB_BACKEND_URI", "http://127.0.0.1:5989"),
couchdbProxyUri: readEnvString("BENCH_COUCHDB_URI", "http://127.0.0.1:15989"),
couchdbUser: readEnvString("BENCH_COUCHDB_USER", readEnvString("username", "admin")),
couchdbPassword: readEnvString("BENCH_COUCHDB_PASSWORD", readEnvString("password", "password")),
couchdbDbname: readEnvString("BENCH_COUCHDB_DBNAME", `bench-couchdb-${Date.now()}`),
datasetDirName: readEnvString("BENCH_DATASET_DIR", "bench-dataset"),
datasetSeed: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
mdFileCount: Math.floor(readEnvNumber("BENCH_MD_FILE_COUNT", 1500)),
mdMinSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MIN_SIZE_BYTES", 1024)),
mdMaxSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MAX_SIZE_BYTES", 20 * 1024)),
binFileCount: Math.floor(readEnvNumber("BENCH_BIN_FILE_COUNT", 500)),
binSizeBytes: Math.floor(readEnvNumber("BENCH_BIN_SIZE_BYTES", 100 * 1024)),
syncTimeoutSeconds: readEnvNumber("BENCH_SYNC_TIMEOUT", 240),
requestedRttMs: Math.floor(readEnvNumber("BENCH_COUCHDB_RTT_MS", 50)),
passphrase: readEnvString("BENCH_PASSPHRASE", `bench-${Date.now()}`),
encrypt: readEnvBool("BENCH_ENCRYPT", true),
};
}
function readOptionalResultPath(): string | undefined {
const raw = Deno.env.get("BENCH_RESULT_JSON")?.trim();
if (!raw) {
return undefined;
}
return raw;
}
function pickSampleFiles(entries: DatasetEntry[]): DatasetEntry[] {
if (entries.length === 0) {
return [];
}
const md = entries.find((e) => e.kind === "md");
const bin = entries.find((e) => e.kind === "bin");
const middle = entries[Math.floor(entries.length / 2)];
const last = entries[entries.length - 1];
const unique = new Map<string, DatasetEntry>();
for (const entry of [md, bin, middle, last]) {
if (entry) {
unique.set(entry.relativePath, entry);
}
}
return [...unique.values()];
}
type ProxyHandle = {
stop: () => Promise<void>;
applied: boolean;
note: string;
};
function startCouchdbProxy(options: { backendUri: string; proxyUri: string; requestedRttMs: number }): ProxyHandle {
const backend = new URL(options.backendUri);
const proxy = new URL(options.proxyUri);
const halfDelayMs = Math.max(1, Math.floor(options.requestedRttMs / 2));
const controller = new AbortController();
const listener = Deno.serve(
{
hostname: proxy.hostname,
port: Number(proxy.port),
signal: controller.signal,
onError(error) {
console.error(`[Proxy] ${String(error)}`);
return new Response("proxy error", { status: 502 });
},
},
async (request) => {
await new Promise((resolve) => setTimeout(resolve, halfDelayMs));
const targetUrl = new URL(request.url);
targetUrl.protocol = backend.protocol;
targetUrl.host = backend.host;
const headers = new Headers(request.headers);
headers.delete("host");
headers.delete("content-length");
let requestBody: ArrayBuffer | undefined;
if (request.method !== "GET" && request.method !== "HEAD") {
try {
requestBody = await request.arrayBuffer();
} catch {
requestBody = undefined;
}
}
const upstream = await fetch(targetUrl, {
method: request.method,
headers,
body: requestBody,
redirect: "manual",
});
const responseHeaders = new Headers(upstream.headers);
responseHeaders.delete("content-length");
const responseBody = await upstream.arrayBuffer();
return new Response(responseBody, {
status: upstream.status,
statusText: upstream.statusText,
headers: responseHeaders,
});
}
);
return {
applied: true,
note: `local reverse proxy on ${proxy.origin} with ${halfDelayMs}ms pre-forward delay`,
stop: async () => {
controller.abort();
await listener.finished.catch(() => {});
},
};
}
async function main(): Promise<void> {
const config = buildConfig();
const resultPath = readOptionalResultPath();
await using workDir = await TempDir.create("livesync-cli-couchdb-bench");
const vaultA = workDir.join("vault-a");
const vaultB = workDir.join("vault-b");
const settingsA = workDir.join("settings-a.json");
const settingsB = workDir.join("settings-b.json");
await Deno.mkdir(vaultA, { recursive: true });
await Deno.mkdir(vaultB, { recursive: true });
await initSettingsFile(settingsA);
await initSettingsFile(settingsB);
await startCouchdb(config.couchdbBackendUri, config.couchdbUser, config.couchdbPassword, config.couchdbDbname);
const proxy = startCouchdbProxy({
backendUri: config.couchdbBackendUri,
proxyUri: config.couchdbProxyUri,
requestedRttMs: config.requestedRttMs,
});
try {
await Promise.all([
applyRemoteSyncSettings(settingsA, {
remoteType: "COUCHDB",
couchdbUri: config.couchdbProxyUri,
couchdbUser: config.couchdbUser,
couchdbPassword: config.couchdbPassword,
couchdbDbname: config.couchdbDbname,
encrypt: config.encrypt,
passphrase: config.passphrase,
}),
applyRemoteSyncSettings(settingsB, {
remoteType: "COUCHDB",
couchdbUri: config.couchdbProxyUri,
couchdbUser: config.couchdbUser,
couchdbPassword: config.couchdbPassword,
couchdbDbname: config.couchdbDbname,
encrypt: config.encrypt,
passphrase: config.passphrase,
}),
]);
const seedFiles = await createDeterministicDataset({
rootDir: vaultA,
datasetDirName: config.datasetDirName,
seed: config.datasetSeed,
mdCount: config.mdFileCount,
mdMinSizeBytes: config.mdMinSizeBytes,
mdMaxSizeBytes: config.mdMaxSizeBytes,
binCount: config.binFileCount,
binSizeBytes: config.binSizeBytes,
});
const mirrorStart = nowMs();
await runCliOrFail(vaultA, "--settings", settingsA, "mirror");
const mirrorElapsed = nowMs() - mirrorStart;
const syncAStart = nowMs();
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
const syncAElapsed = nowMs() - syncAStart;
const syncBStart = nowMs();
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
const syncBElapsed = nowMs() - syncBStart;
const sampleFiles = pickSampleFiles(seedFiles.entries);
for (const sample of sampleFiles) {
const pulledPath = workDir.join(`pulled-${sample.relativePath.split("/").join("_")}`);
await runCliOrFail(vaultB, "--settings", settingsB, "pull", sample.relativePath, pulledPath);
await assertFilesEqual(
sample.absolutePath,
pulledPath,
`sample file mismatch after CouchDB sync: ${sample.relativePath}`
);
}
const result = {
mode: "couchdb-cli-benchmark",
couchdbBackendUri: config.couchdbBackendUri,
couchdbProxyUri: config.couchdbProxyUri,
couchdbDbname: config.couchdbDbname,
rttRequestedMs: config.requestedRttMs,
proxyApplied: proxy.applied,
proxyNote: proxy.note,
datasetSeed: config.datasetSeed,
datasetDirName: config.datasetDirName,
totalFiles: seedFiles.totalFiles,
totalBytes: seedFiles.totalBytes,
mdFileCount: seedFiles.mdCount,
binFileCount: seedFiles.binCount,
mirrorElapsedMs: Number(mirrorElapsed.toFixed(1)),
syncAElapsedMs: Number(syncAElapsed.toFixed(1)),
syncBElapsedMs: Number(syncBElapsed.toFixed(1)),
totalSyncElapsedMs: Number((syncAElapsed + syncBElapsed).toFixed(1)),
throughputBytesPerSec: Number((seedFiles.totalBytes / ((syncAElapsed + syncBElapsed) / 1000)).toFixed(2)),
throughputMiBPerSec: Number(
(seedFiles.totalBytes / ((syncAElapsed + syncBElapsed) / 1000) / 1024 / 1024).toFixed(4)
),
};
if (resultPath) {
await Deno.writeTextFile(resultPath, JSON.stringify(result, null, 2));
}
console.log(JSON.stringify(result, null, 2));
console.error(
`[Benchmark] couchdb mirrored ${seedFiles.totalFiles} files (${formatBytes(seedFiles.totalBytes)}) in ${formatMs(
mirrorElapsed
)}, synced in ${formatMs(syncAElapsed + syncBElapsed)} (${result.throughputBytesPerSec} B/s, ${result.throughputMiBPerSec} MiB/s)`
);
} finally {
await proxy.stop();
await stopCouchdb().catch(() => {});
}
}
if (import.meta.main) {
main().catch((error) => {
console.error(`[Fatal Error]`, error);
Deno.exit(1);
});
}

View File

@@ -0,0 +1,223 @@
import { TempDir } from "./helpers/temp.ts";
import { applyP2pSettings, applyP2pTestTweaks, initSettingsFile } from "./helpers/settings.ts";
import { startCliInBackground } from "./helpers/backgroundCli.ts";
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
import { createDeterministicDataset, type DatasetEntry } from "./helpers/dataset.ts";
type BenchmarkConfig = {
relay: string;
appId: string;
roomId: string;
passphrase: string;
datasetDirName: string;
datasetSeed: string;
mdFileCount: number;
mdMinSizeBytes: number;
mdMaxSizeBytes: number;
binFileCount: number;
binSizeBytes: number;
peersTimeoutSeconds: number;
syncTimeoutSeconds: number;
};
function readEnvString(name: string, fallback: string): string {
const value = Deno.env.get(name)?.trim();
return value && value.length > 0 ? value : fallback;
}
function readEnvNumber(name: string, fallback: number): number {
const raw = Deno.env.get(name);
if (raw === undefined || raw.trim() === "") {
return fallback;
}
const parsed = Number(raw);
if (!Number.isFinite(parsed) || parsed <= 0) {
throw new Error(`${name} must be a positive number, got '${raw}'`);
}
return parsed;
}
function nowMs(): number {
return performance.now();
}
function formatMs(value: number): string {
return `${value.toFixed(1)} ms`;
}
function formatBytes(value: number): string {
if (value < 1024) {
return `${value} B`;
}
const kib = value / 1024;
if (kib < 1024) {
return `${kib.toFixed(1)} KiB`;
}
const mib = kib / 1024;
return `${mib.toFixed(1)} MiB`;
}
function buildConfig(): BenchmarkConfig {
return {
relay: readEnvString("BENCH_RELAY", "ws://localhost:4000/"),
appId: readEnvString("BENCH_APP_ID", "self-hosted-livesync-cli-benchmark"),
roomId: readEnvString("BENCH_ROOM_ID", `bench-room-${Date.now()}`),
passphrase: readEnvString("BENCH_PASSPHRASE", `bench-${Date.now()}`),
datasetDirName: readEnvString("BENCH_DATASET_DIR", "bench-dataset"),
datasetSeed: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
mdFileCount: Math.floor(readEnvNumber("BENCH_MD_FILE_COUNT", 1500)),
mdMinSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MIN_SIZE_BYTES", 1024)),
mdMaxSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MAX_SIZE_BYTES", 20 * 1024)),
binFileCount: Math.floor(readEnvNumber("BENCH_BIN_FILE_COUNT", 500)),
binSizeBytes: Math.floor(readEnvNumber("BENCH_BIN_SIZE_BYTES", 100 * 1024)),
peersTimeoutSeconds: readEnvNumber("BENCH_PEERS_TIMEOUT", 20),
syncTimeoutSeconds: readEnvNumber("BENCH_SYNC_TIMEOUT", 240),
};
}
function readOptionalResultPath(): string | undefined {
const raw = Deno.env.get("BENCH_RESULT_JSON")?.trim();
if (!raw) {
return undefined;
}
return raw;
}
function pickSampleFiles(entries: DatasetEntry[]): DatasetEntry[] {
if (entries.length === 0) {
return [];
}
const md = entries.find((e) => e.kind === "md");
const bin = entries.find((e) => e.kind === "bin");
const middle = entries[Math.floor(entries.length / 2)];
const last = entries[entries.length - 1];
const unique = new Map<string, DatasetEntry>();
for (const entry of [md, bin, middle, last]) {
if (entry) {
unique.set(entry.relativePath, entry);
}
}
return [...unique.values()];
}
async function main(): Promise<void> {
const config = buildConfig();
const resultPath = readOptionalResultPath();
const relayStarted = await maybeStartLocalRelay(config.relay);
await using workDir = await TempDir.create("livesync-cli-p2p-bench");
const hostVault = workDir.join("vault-host");
const clientVault = workDir.join("vault-client");
const hostSettings = workDir.join("settings-host.json");
const clientSettings = workDir.join("settings-client.json");
await Promise.all([
Deno.mkdir(hostVault, { recursive: true }),
Deno.mkdir(clientVault, { recursive: true }),
initSettingsFile(hostSettings),
initSettingsFile(clientSettings),
]);
await Promise.all([
applyP2pSettings(hostSettings, config.roomId, config.passphrase, config.appId, config.relay, "~.*"),
applyP2pSettings(clientSettings, config.roomId, config.passphrase, config.appId, config.relay, "~.*"),
]);
await Promise.all([
applyP2pTestTweaks(hostSettings, "p2p-bench-host", config.passphrase),
applyP2pTestTweaks(clientSettings, "p2p-bench-client", config.passphrase),
]);
const seedFiles = await createDeterministicDataset({
rootDir: hostVault,
datasetDirName: config.datasetDirName,
seed: config.datasetSeed,
mdCount: config.mdFileCount,
mdMinSizeBytes: config.mdMinSizeBytes,
mdMaxSizeBytes: config.mdMaxSizeBytes,
binCount: config.binFileCount,
binSizeBytes: config.binSizeBytes,
});
const mirrorStart = nowMs();
await runCliOrFail(hostVault, "--settings", hostSettings, "mirror");
const mirrorElapsed = nowMs() - mirrorStart;
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
try {
const hostReadyStart = nowMs();
await host.waitUntilContains("P2P host is running", 20000);
const hostReadyElapsed = nowMs() - hostReadyStart;
const peerDiscoveryStart = nowMs();
const peer = await discoverPeer(clientVault, clientSettings, config.peersTimeoutSeconds);
const peerDiscoveryElapsed = nowMs() - peerDiscoveryStart;
const syncStart = nowMs();
await runCliOrFail(
clientVault,
"--settings",
clientSettings,
"p2p-sync",
peer.id,
String(config.syncTimeoutSeconds)
);
const syncElapsed = nowMs() - syncStart;
const sampleFiles = pickSampleFiles(seedFiles.entries);
for (const sample of sampleFiles) {
const pulledPath = workDir.join(`pulled-${sample.relativePath.replaceAll("/", "_")}`);
await runCliOrFail(clientVault, "--settings", clientSettings, "pull", sample.relativePath, pulledPath);
await assertFilesEqual(
sample.absolutePath,
pulledPath,
`sample file mismatch after sync: ${sample.relativePath}`
);
}
const result = {
mode: "p2p-cli-benchmark",
relay: config.relay,
appId: config.appId,
roomId: config.roomId,
datasetSeed: config.datasetSeed,
datasetDirName: config.datasetDirName,
peerId: peer.id,
peerName: peer.name,
totalFiles: seedFiles.totalFiles,
totalBytes: seedFiles.totalBytes,
mdFileCount: seedFiles.mdCount,
binFileCount: seedFiles.binCount,
mirrorElapsedMs: Number(mirrorElapsed.toFixed(1)),
hostReadyElapsedMs: Number(hostReadyElapsed.toFixed(1)),
peerDiscoveryElapsedMs: Number(peerDiscoveryElapsed.toFixed(1)),
syncElapsedMs: Number(syncElapsed.toFixed(1)),
throughputBytesPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000)).toFixed(2)),
throughputMiBPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000) / 1024 / 1024).toFixed(4)),
};
if (resultPath) {
await Deno.writeTextFile(resultPath, JSON.stringify(result, null, 2));
}
console.log(JSON.stringify(result, null, 2));
console.error(
`[Benchmark] mirrored ${seedFiles.totalFiles} files (${formatBytes(seedFiles.totalBytes)}) in ${formatMs(mirrorElapsed)}, ` +
`synced in ${formatMs(syncElapsed)} ` +
`(${result.throughputBytesPerSec} B/s, ${result.throughputMiBPerSec} MiB/s)`
);
} finally {
await host.stop();
await stopLocalRelayIfStarted(relayStarted);
}
}
if (import.meta.main) {
main().catch((error) => {
console.error(`[Fatal Error]`, error);
Deno.exit(1);
});
}

View File

@@ -0,0 +1,45 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
RESULTS_ROOT="${SCRIPT_DIR}/bench-results"
TIMESTAMP="$(date +%Y%m%d-%H%M%S)"
OUT_DIR="${RESULTS_ROOT}/${TIMESTAMP}"
mkdir -p "${OUT_DIR}"
echo "[bench-wrapper] output directory: ${OUT_DIR}"
echo "[bench-wrapper] running p2p benchmark"
(
cd "${SCRIPT_DIR}"
BENCH_RESULT_JSON="${OUT_DIR}/p2p.json" deno task bench:p2p
)
echo "[bench-wrapper] running couchdb benchmark with RTT ${BENCH_COUCHDB_RTT_MS:-default} ms (emulating HTTP network latency)"
(
cd "${SCRIPT_DIR}"
BENCH_RESULT_JSON="${OUT_DIR}/couchdb.json" deno task bench:couchdb
)
cat > "${OUT_DIR}/README.txt" <<EOF
Bench wrapper result set
Generated at: ${TIMESTAMP}
Directory: ${OUT_DIR}
Files:
- p2p.json
- couchdb.json
EOF
echo "[bench-wrapper] verify outputs by cat"
echo "========== ${OUT_DIR}/README.txt =========="
cat "${OUT_DIR}/README.txt"
echo "========== ${OUT_DIR}/p2p.json =========="
cat "${OUT_DIR}/p2p.json"
echo "========== ${OUT_DIR}/couchdb.json =========="
cat "${OUT_DIR}/couchdb.json"
echo "[bench-wrapper] done"
echo "[bench-wrapper] result directory: ${OUT_DIR}"

View File

@@ -12,8 +12,16 @@
"test:p2p-sync": "deno test --env-file=.test.env -A --no-check test-p2p-sync.ts",
"test:p2p-three-nodes": "deno test --env-file=.test.env -A --no-check test-p2p-three-nodes-conflict.ts",
"test:p2p-upload-download": "deno test --env-file=.test.env -A --no-check test-p2p-upload-download-repro.ts",
"bench:p2p": "deno run --env-file=.test.env -A --no-check bench-p2p.ts",
"bench:couchdb": "deno run --env-file=.test.env -A --no-check bench-couchdb.ts",
"bench:item1": "bash ./bench-run-item1.sh",
"bench:item1:full": "BENCH_MD_FILE_COUNT=1500 BENCH_MD_MIN_SIZE_BYTES=1024 BENCH_MD_MAX_SIZE_BYTES=20480 BENCH_BIN_FILE_COUNT=500 BENCH_BIN_SIZE_BYTES=102400 BENCH_COUCHDB_RTT_MS=50 bash ./bench-run-item1.sh",
"test:e2e-couchdb": "deno test --env-file=.test.env -A --no-check test-e2e-two-vaults-couchdb.ts",
"test:e2e-matrix": "deno test --env-file=.test.env -A --no-check test-e2e-two-vaults-matrix.ts"
"test:e2e-matrix": "deno test --env-file=.test.env -A --no-check test-e2e-two-vaults-matrix.ts",
"test:e2e-matrix:couchdb-enc0": "deno test --env-file=.test.env -A --no-check --filter='e2e matrix: COUCHDB-enc0' test-e2e-two-vaults-matrix.ts",
"test:e2e-matrix:couchdb-enc1": "deno test --env-file=.test.env -A --no-check --filter='e2e matrix: COUCHDB-enc1' test-e2e-two-vaults-matrix.ts",
"test:e2e-matrix:minio-enc0": "deno test --env-file=.test.env -A --no-check --filter='e2e matrix: MINIO-enc0' test-e2e-two-vaults-matrix.ts",
"test:e2e-matrix:minio-enc1": "deno test --env-file=.test.env -A --no-check --filter='e2e matrix: MINIO-enc1' test-e2e-two-vaults-matrix.ts"
},
"imports": {
"@std/assert": "jsr:@std/assert@^1.0.13",

View File

@@ -39,27 +39,73 @@ function concatChunks(chunks: Uint8Array[]): Uint8Array {
return out;
}
function formatTeeCommand(args: string[]): string {
return ["node", CLI_DIST, ...args].map((part) => JSON.stringify(part)).join(" ");
}
function createLineTeeWriter(
pid: number,
streamName: "stdout" | "stderr",
writer: (chunk: Uint8Array) => void
): { write: (chunk: Uint8Array) => void; close: () => void } {
const enc = new TextEncoder();
const dec = new TextDecoder();
let pending = "";
let headerWritten = false;
const emitLine = (line: string) => {
if (!headerWritten) {
writer(enc.encode(`[CLI tee pid=${pid}:${streamName}]\n`));
headerWritten = true;
}
writer(enc.encode(`[CLI tee pid=${pid}:${streamName}] ${line}\n`));
};
const flush = (final = false) => {
let index = pending.indexOf("\n");
while (index >= 0) {
const line = pending.slice(0, index).replace(/\r$/, "");
pending = pending.slice(index + 1);
emitLine(line);
index = pending.indexOf("\n");
}
if (final && pending.length > 0) {
emitLine(pending.replace(/\r$/, ""));
pending = "";
}
};
return {
write(chunk: Uint8Array) {
pending += dec.decode(chunk, { stream: true });
flush(false);
},
close() {
pending += dec.decode();
flush(true);
},
};
}
async function collectStream(
stream: ReadableStream<Uint8Array>,
teeTarget: WritableStream<Uint8Array> | null
teeTarget: { write: (chunk: Uint8Array) => void; close: () => void } | null
): Promise<Uint8Array> {
const reader = stream.getReader();
const chunks: Uint8Array[] = [];
const writer = teeTarget?.getWriter();
try {
while (true) {
const { done, value } = await reader.read();
if (done) break;
if (value) {
chunks.push(value);
if (writer) {
await writer.write(value);
if (teeTarget) {
teeTarget.write(value);
}
}
}
} finally {
if (writer) {
writer.releaseLock();
if (teeTarget) {
teeTarget.close();
}
reader.releaseLock();
}
@@ -76,8 +122,20 @@ async function runNodeCommand(args: string[], stdinData?: Uint8Array): Promise<C
stderr: "piped",
}).spawn();
const stdoutPromise = collectStream(child.stdout, TEE_ENABLED ? Deno.stdout.writable : null);
const stderrPromise = collectStream(child.stderr, TEE_ENABLED ? Deno.stderr.writable : null);
if (TEE_ENABLED) {
Deno.stdout.writeSync(
new TextEncoder().encode(`[CLI tee pid=${child.pid}] process: ${formatTeeCommand(cliArgs)}\n`)
);
}
const stdoutPromise = collectStream(
child.stdout,
TEE_ENABLED ? createLineTeeWriter(child.pid, "stdout", (chunk) => Deno.stdout.writeSync(chunk)) : null
);
const stderrPromise = collectStream(
child.stderr,
TEE_ENABLED ? createLineTeeWriter(child.pid, "stderr", (chunk) => Deno.stderr.writeSync(chunk)) : null
);
if (stdinData) {
const w = child.stdin.getWriter();

View File

@@ -0,0 +1,123 @@
export type DeterministicDatasetConfig = {
rootDir: string;
datasetDirName: string;
seed: string;
mdCount: number;
mdMinSizeBytes: number;
mdMaxSizeBytes: number;
binCount: number;
binSizeBytes: number;
};
export type DatasetEntry = {
kind: "md" | "bin";
relativePath: string;
absolutePath: string;
size: number;
};
export type DeterministicDataset = {
rootDir: string;
datasetDirName: string;
seed: string;
entries: DatasetEntry[];
totalFiles: number;
totalBytes: number;
mdCount: number;
binCount: number;
};
function fnv1a32(input: string): number {
let hash = 0x811c9dc5;
for (let i = 0; i < input.length; i++) {
hash ^= input.charCodeAt(i) & 0xff;
hash = Math.imul(hash, 0x01000193);
}
return hash >>> 0;
}
function createXorshift32(seed: number): () => number {
let state = seed >>> 0;
if (state === 0) {
state = 0x9e3779b9;
}
return () => {
state ^= state << 13;
state ^= state >>> 17;
state ^= state << 5;
return state >>> 0;
};
}
function createTextBytes(size: number, fileIndex: number, seed: string): Uint8Array {
const template =
`# Bench file ${fileIndex}\n` +
`seed: ${seed}\n` +
"lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.\n";
const templateBytes = new TextEncoder().encode(template);
const out = new Uint8Array(size);
for (let i = 0; i < size; i++) {
out[i] = templateBytes[i % templateBytes.length];
}
return out;
}
function toPath(rootDir: string, relativePath: string): string {
return `${rootDir}/${relativePath}`;
}
export async function createDeterministicDataset(config: DeterministicDatasetConfig): Promise<DeterministicDataset> {
if (config.mdCount < 0 || config.binCount < 0) {
throw new Error("mdCount and binCount must be non-negative");
}
if (config.mdMinSizeBytes <= 0 || config.mdMaxSizeBytes <= 0 || config.binSizeBytes <= 0) {
throw new Error("all size values must be positive");
}
if (config.mdMinSizeBytes > config.mdMaxSizeBytes) {
throw new Error("mdMinSizeBytes must be <= mdMaxSizeBytes");
}
const datasetRoot = toPath(config.rootDir, config.datasetDirName);
const mdDir = `${datasetRoot}/md`;
const binDir = `${datasetRoot}/bin`;
await Deno.mkdir(mdDir, { recursive: true });
await Deno.mkdir(binDir, { recursive: true });
const nextRandom = createXorshift32(fnv1a32(config.seed));
const mdRange = config.mdMaxSizeBytes - config.mdMinSizeBytes + 1;
const entries: DatasetEntry[] = [];
for (let index = 0; index < config.mdCount; index++) {
const size = config.mdMinSizeBytes + (nextRandom() % mdRange);
const relativePath = `${config.datasetDirName}/md/file-${String(index).padStart(4, "0")}.md`;
const absolutePath = toPath(config.rootDir, relativePath);
const body = createTextBytes(size, index, config.seed);
await Deno.writeFile(absolutePath, body);
entries.push({ kind: "md", relativePath, absolutePath, size });
}
for (let index = 0; index < config.binCount; index++) {
const size = config.binSizeBytes;
const relativePath = `${config.datasetDirName}/bin/file-${String(index).padStart(4, "0")}.bin`;
const absolutePath = toPath(config.rootDir, relativePath);
const body = new Uint8Array(size);
for (let i = 0; i < size; i++) {
body[i] = nextRandom() & 0xff;
}
await Deno.writeFile(absolutePath, body);
entries.push({ kind: "bin", relativePath, absolutePath, size });
}
const totalBytes = entries.reduce((sum, e) => sum + e.size, 0);
return {
rootDir: config.rootDir,
datasetDirName: config.datasetDirName,
seed: config.seed,
entries,
totalFiles: entries.length,
totalBytes,
mdCount: config.mdCount,
binCount: config.binCount,
};
}

View File

@@ -14,6 +14,11 @@ type DockerInvoker = {
let dockerInvokerPromise: Promise<DockerInvoker> | null = null;
const DOCKER_TEE = Deno.env.get("LIVESYNC_DOCKER_TEE") === "1" || Deno.env.get("LIVESYNC_TEST_TEE") === "1";
const trackedContainers = new Set<string>();
const CLEANUP_SIGNALS: Deno.Signal[] = ["SIGINT", "SIGTERM"];
let signalCleanupHandlersInstalled = false;
let signalCleanupInProgress = false;
const signalCleanupHandlers = new Map<Deno.Signal, () => void>();
// ---------------------------------------------------------------------------
// Low-level docker wrapper
@@ -27,29 +32,53 @@ function parseCommand(command: string): { bin: string; prefix: string[] } {
return { bin: parts[0], prefix: parts.slice(1) };
}
async function runCommand(bin: string, args: string[]): Promise<{ code: number; stdout: string; stderr: string }> {
const cmd = new Deno.Command(bin, {
args,
stdin: "null",
stdout: "piped",
stderr: "piped",
});
async function collectStream(
stream: ReadableStream<Uint8Array>,
teeTarget: ((chunk: Uint8Array) => void) | null
): Promise<Uint8Array> {
const reader = stream.getReader();
const chunks: Uint8Array[] = [];
try {
const { code, stdout, stderr } = await cmd.output();
while (true) {
const { done, value } = await reader.read();
if (done) break;
if (!value) continue;
chunks.push(value);
if (teeTarget) {
teeTarget(value);
}
}
} finally {
reader.releaseLock();
}
const total = chunks.reduce((sum, chunk) => sum + chunk.length, 0);
const out = new Uint8Array(total);
let offset = 0;
for (const chunk of chunks) {
out.set(chunk, offset);
offset += chunk.length;
}
return out;
}
async function runCommand(bin: string, args: string[]): Promise<{ code: number; stdout: string; stderr: string }> {
try {
const child = new Deno.Command(bin, {
args,
stdin: "null",
stdout: "piped",
stderr: "piped",
}).spawn();
const stdoutPromise = collectStream(child.stdout, DOCKER_TEE ? (chunk) => Deno.stdout.writeSync(chunk) : null);
const stderrPromise = collectStream(child.stderr, DOCKER_TEE ? (chunk) => Deno.stderr.writeSync(chunk) : null);
const [status, stdout, stderr] = await Promise.all([child.status, stdoutPromise, stderrPromise]);
const dec = new TextDecoder();
const result = {
code,
code: status.code,
stdout: dec.decode(stdout),
stderr: dec.decode(stderr),
};
if (DOCKER_TEE) {
if (result.stdout.trim().length > 0) {
console.log(`[docker:${bin}] ${result.stdout.trimEnd()}`);
}
if (result.stderr.trim().length > 0) {
console.error(`[docker:${bin}] ${result.stderr.trimEnd()}`);
}
}
return result;
} catch (err) {
if (err instanceof Deno.errors.NotFound) {
@@ -159,6 +188,73 @@ async function dockerOrFail(...args: string[]): Promise<string> {
return r.stdout;
}
async function stopAndRemoveContainer(container: string): Promise<void> {
await docker("stop", container).catch(() => {});
await docker("rm", container).catch(() => {});
}
async function cleanupTrackedContainers(reason: string): Promise<void> {
const names = [...trackedContainers];
if (names.length === 0) return;
console.warn(`[WARN] cleaning up tracked containers on ${reason}: ${names.join(", ")}`);
for (const container of names.reverse()) {
await stopAndRemoveContainer(container);
trackedContainers.delete(container);
}
}
async function handleSignalCleanup(signal: Deno.Signal): Promise<void> {
if (signalCleanupInProgress) return;
signalCleanupInProgress = true;
try {
await cleanupTrackedContainers(`signal ${signal}`);
} finally {
Deno.exit(signal === "SIGINT" ? 130 : 143);
}
}
function ensureSignalCleanupHandlers(): void {
if (signalCleanupHandlersInstalled) return;
signalCleanupHandlersInstalled = true;
for (const signal of CLEANUP_SIGNALS) {
const listener = () => {
void handleSignalCleanup(signal);
};
try {
Deno.addSignalListener(signal, listener);
signalCleanupHandlers.set(signal, listener);
} catch {
// Unsupported signal on this platform.
}
}
}
function removeSignalCleanupHandlers(): void {
if (!signalCleanupHandlersInstalled) return;
for (const [signal, listener] of signalCleanupHandlers) {
try {
Deno.removeSignalListener(signal, listener);
} catch {
// Ignore if already removed or unsupported.
}
}
signalCleanupHandlers.clear();
signalCleanupHandlersInstalled = false;
}
function trackContainer(container: string): void {
ensureSignalCleanupHandlers();
trackedContainers.add(container);
}
function untrackContainer(container: string): void {
trackedContainers.delete(container);
if (trackedContainers.size === 0) {
removeSignalCleanupHandlers();
}
}
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
@@ -235,8 +331,8 @@ const MINIO_IMAGE = "minio/minio";
const MINIO_MC_IMAGE = "minio/mc";
export async function stopCouchdb(): Promise<void> {
await docker("stop", COUCHDB_CONTAINER);
await docker("rm", COUCHDB_CONTAINER);
await stopAndRemoveContainer(COUCHDB_CONTAINER);
untrackContainer(COUCHDB_CONTAINER);
}
/**
@@ -265,6 +361,7 @@ export async function startCouchdb(couchdbUri: string, user: string, password: s
"COUCHDB_SINGLE_NODE=y",
COUCHDB_IMAGE
);
trackContainer(COUCHDB_CONTAINER);
console.log("[INFO] initialising CouchDB");
await initCouchdb(couchdbUri, user, password);
@@ -365,8 +462,8 @@ function shQuote(value: string): string {
}
export async function stopMinio(): Promise<void> {
await docker("stop", MINIO_CONTAINER);
await docker("rm", MINIO_CONTAINER);
await stopAndRemoveContainer(MINIO_CONTAINER);
untrackContainer(MINIO_CONTAINER);
}
async function initMinioBucket(
@@ -446,6 +543,7 @@ export async function startMinio(
"--console-address",
":9001"
);
trackContainer(MINIO_CONTAINER);
console.log(`[INFO] initialising MinIO test bucket: ${bucket}`);
let initialised = false;
@@ -493,8 +591,8 @@ EOF
exec /app/strfry --config /tmp/strfry.conf relay`;
export async function stopP2pRelay(): Promise<void> {
await docker("stop", P2P_RELAY_CONTAINER);
await docker("rm", P2P_RELAY_CONTAINER);
await stopAndRemoveContainer(P2P_RELAY_CONTAINER);
untrackContainer(P2P_RELAY_CONTAINER);
}
/**
@@ -523,6 +621,7 @@ export async function startP2pRelay(): Promise<void> {
"-lc",
STRFRY_BOOTSTRAP_SH
);
trackContainer(P2P_RELAY_CONTAINER);
}
export function isLocalP2pRelay(relayUrl: string): boolean {

View File

@@ -0,0 +1,49 @@
type WaitForPortOptions = {
timeoutMs?: number;
intervalMs?: number;
connectTimeoutMs?: number;
};
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function connectWithTimeout(hostname: string, port: number, timeoutMs: number): Promise<void> {
let timer: number | undefined;
try {
const connPromise = Deno.connect({ hostname, port });
const timeoutPromise = new Promise<never>((_, reject) => {
timer = setTimeout(() => reject(new Error(`connect timeout after ${timeoutMs}ms`)), timeoutMs);
});
const conn = await Promise.race([connPromise, timeoutPromise]);
conn.close();
} finally {
if (timer !== undefined) {
clearTimeout(timer);
}
}
}
export async function waitForPort(hostname: string, port: number, options: WaitForPortOptions = {}): Promise<void> {
const timeoutMs = options.timeoutMs ?? 15000;
const intervalMs = options.intervalMs ?? 250;
const connectTimeoutMs = options.connectTimeoutMs ?? 1000;
const started = Date.now();
let lastError: unknown;
while (Date.now() - started < timeoutMs) {
try {
await connectWithTimeout(hostname, port, connectTimeoutMs);
return;
} catch (error) {
lastError = error;
await sleep(intervalMs);
}
}
throw new Error(
`Port ${hostname}:${port} did not become ready within ${timeoutMs}ms` +
(lastError ? ` (last error: ${String(lastError)})` : "")
);
}

View File

@@ -1,11 +1,26 @@
import { runCli } from "./cli.ts";
import { isLocalP2pRelay, startP2pRelay, stopP2pRelay } from "./docker.ts";
import { waitForPort } from "./net.ts";
export type PeerEntry = {
id: string;
name: string;
};
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function parseRelayEndpoint(relay: string): { hostname: string; port: number } {
const url = new URL(relay);
const port = url.port ? Number(url.port) : url.protocol === "ws:" ? 80 : url.protocol === "wss:" ? 443 : NaN;
if (!Number.isFinite(port)) {
throw new Error(`Unsupported relay URL: ${relay}`);
}
const hostname = url.hostname === "localhost" ? "127.0.0.1" : url.hostname;
return { hostname, port };
}
export function parsePeerLines(output: string): PeerEntry[] {
return output
.split(/\r?\n/)
@@ -20,28 +35,55 @@ export async function discoverPeer(
timeoutSeconds: number,
targetPeer?: string
): Promise<PeerEntry> {
const result = await runCli(vaultDir, "--settings", settingsFile, "p2p-peers", String(timeoutSeconds));
if (result.code !== 0) {
throw new Error(`p2p-peers failed\n${result.combined}`);
}
const peers = parsePeerLines(result.stdout);
if (targetPeer) {
const matched = peers.find((peer) => peer.id === targetPeer || peer.name === targetPeer);
if (matched) return matched;
}
if (peers.length === 0) {
const fallback = result.combined.match(/Advertisement from\s+([^\s]+)/);
if (fallback?.[1]) {
return { id: fallback[1], name: fallback[1] };
const retries = Math.max(0, Number(Deno.env.get("LIVESYNC_P2P_PEERS_RETRY") ?? "3"));
let lastCombined = "";
for (let attempt = 0; attempt <= retries; attempt++) {
const result = await runCli(vaultDir, "--settings", settingsFile, "p2p-peers", String(timeoutSeconds));
lastCombined = result.combined;
if (result.code === 0) {
const peers = parsePeerLines(result.stdout);
if (targetPeer) {
const matched = peers.find((peer) => peer.id === targetPeer || peer.name === targetPeer);
if (matched) return matched;
}
if (peers.length > 0) {
return peers[0];
}
const fallback = result.combined.match(/Advertisement from\s+([^\s]+)/);
if (fallback?.[1]) {
return { id: fallback[1], name: fallback[1] };
}
}
throw new Error(`No peers discovered\n${result.combined}`);
if (attempt < retries) {
const waitMs = 400 * (attempt + 1);
console.warn(
`[WARN] p2p-peers returned no usable peers, retrying (${attempt + 1}/${retries}) in ${waitMs}ms`
);
await sleep(waitMs);
continue;
}
throw new Error(
result.code !== 0 ? `p2p-peers failed\n${result.combined}` : `No peers discovered\n${result.combined}`
);
}
return peers[0];
throw new Error(`No peers discovered\n${lastCombined}`);
}
export async function maybeStartLocalRelay(relay: string): Promise<boolean> {
if (!isLocalP2pRelay(relay)) return false;
await startP2pRelay();
const endpoint = parseRelayEndpoint(relay);
await waitForPort(endpoint.hostname, endpoint.port, {
timeoutMs: Number(Deno.env.get("LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS") ?? "15000"),
intervalMs: Number(Deno.env.get("LIVESYNC_P2P_RELAY_READY_INTERVAL_MS") ?? "250"),
connectTimeoutMs: Number(Deno.env.get("LIVESYNC_P2P_RELAY_CONNECT_TIMEOUT_MS") ?? "1000"),
});
return true;
}

View File

@@ -9,6 +9,9 @@ Deno.test("p2p-peers: discovers host through local relay", async () => {
const roomId = Deno.env.get("ROOM_ID") ?? `room-${Date.now()}`;
const passphrase = Deno.env.get("PASSPHRASE") ?? "test";
const timeoutSeconds = Number(Deno.env.get("TIMEOUT_SECONDS") ?? "8");
const nonce = `${Date.now()}-${crypto.randomUUID().slice(0, 8)}`;
const hostPeerName = Deno.env.get("HOST_PEER_NAME") ?? `p2p-host-${nonce}`;
const clientPeerName = Deno.env.get("CLIENT_PEER_NAME") ?? `p2p-client-${nonce}`;
await using workDir = await TempDir.create("livesync-cli-p2p-peers-local-relay");
const hostVault = workDir.join("vault-host");
@@ -24,15 +27,16 @@ Deno.test("p2p-peers: discovers host through local relay", async () => {
await initSettingsFile(clientSettings);
await applyP2pSettings(hostSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
await applyP2pSettings(clientSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
await applyP2pTestTweaks(hostSettings, "p2p-host", passphrase);
await applyP2pTestTweaks(clientSettings, "p2p-client", passphrase);
await applyP2pTestTweaks(hostSettings, hostPeerName, passphrase);
await applyP2pTestTweaks(clientSettings, clientPeerName, passphrase);
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
try {
await host.waitUntilContains("P2P host is running", 20000);
const peer = await discoverPeer(clientVault, clientSettings, timeoutSeconds);
const peer = await discoverPeer(clientVault, clientSettings, timeoutSeconds, hostPeerName);
assert(peer.id.length > 0);
assert(peer.name.length > 0);
assert(peer.name === hostPeerName, `expected peer '${hostPeerName}', got '${peer.name}'`);
} finally {
await host.stop();
}

View File

@@ -11,6 +11,9 @@ Deno.test("p2p-sync: discovers peer and completes sync", async () => {
const passphrase = Deno.env.get("PASSPHRASE") ?? "test";
const peersTimeout = Number(Deno.env.get("PEERS_TIMEOUT") ?? "12");
const syncTimeout = Number(Deno.env.get("SYNC_TIMEOUT") ?? "15");
const nonce = `${Date.now()}-${crypto.randomUUID().slice(0, 8)}`;
const hostPeerName = Deno.env.get("HOST_PEER_NAME") ?? `p2p-host-${nonce}`;
const clientPeerName = Deno.env.get("CLIENT_PEER_NAME") ?? `p2p-client-${nonce}`;
await using workDir = await TempDir.create("livesync-cli-p2p-sync");
const hostVault = workDir.join("vault-host");
@@ -26,8 +29,8 @@ Deno.test("p2p-sync: discovers peer and completes sync", async () => {
await initSettingsFile(clientSettings);
await applyP2pSettings(hostSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
await applyP2pSettings(clientSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
await applyP2pTestTweaks(hostSettings, "p2p-host", passphrase);
await applyP2pTestTweaks(clientSettings, "p2p-client", passphrase);
await applyP2pTestTweaks(hostSettings, hostPeerName, passphrase);
await applyP2pTestTweaks(clientSettings, clientPeerName, passphrase);
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
try {
@@ -36,7 +39,7 @@ Deno.test("p2p-sync: discovers peer and completes sync", async () => {
clientVault,
clientSettings,
peersTimeout,
Deno.env.get("TARGET_PEER") ?? undefined
Deno.env.get("TARGET_PEER") ?? hostPeerName
);
const syncResult = await runCli(
clientVault,

View File

@@ -1,6 +1,6 @@
import { assert } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { applyP2pSettings, initSettingsFile } from "./helpers/settings.ts";
import { applyP2pSettings, applyP2pTestTweaks, initSettingsFile } from "./helpers/settings.ts";
import { startCliInBackground } from "./helpers/backgroundCli.ts";
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
import { jsonStringField, runCliOrFail, runCliWithInputOrFail, sanitiseCatStdout } from "./helpers/cli.ts";
@@ -12,6 +12,10 @@ Deno.test("p2p: three nodes detect and resolve conflicts", async () => {
const appId = Deno.env.get("APP_ID") ?? "self-hosted-livesync-cli-tests";
const peersTimeout = Number(Deno.env.get("PEERS_TIMEOUT") ?? "10");
const syncTimeout = Number(Deno.env.get("SYNC_TIMEOUT") ?? "15");
const nonce = `${Date.now()}-${crypto.randomUUID().slice(0, 8)}`;
const hostPeerName = Deno.env.get("HOST_PEER_NAME") ?? `p2p-host-${nonce}`;
const peerNameB = Deno.env.get("PEER_NAME_B") ?? `p2p-client-b-${nonce}`;
const peerNameC = Deno.env.get("PEER_NAME_C") ?? `p2p-client-c-${nonce}`;
await using workDir = await TempDir.create("livesync-cli-p2p-3nodes");
const vaultA = workDir.join("vault-a");
@@ -26,16 +30,21 @@ Deno.test("p2p: three nodes detect and resolve conflicts", async () => {
const relayStarted = await maybeStartLocalRelay(relay);
try {
for (const settings of [settingsA, settingsB, settingsC]) {
await initSettingsFile(settings);
await applyP2pSettings(settings, roomId, passphrase, appId, relay);
}
await initSettingsFile(settingsA);
await initSettingsFile(settingsB);
await initSettingsFile(settingsC);
await applyP2pSettings(settingsA, roomId, passphrase, appId, relay);
await applyP2pSettings(settingsB, roomId, passphrase, appId, relay);
await applyP2pSettings(settingsC, roomId, passphrase, appId, relay);
await applyP2pTestTweaks(settingsA, hostPeerName, passphrase);
await applyP2pTestTweaks(settingsB, peerNameB, passphrase);
await applyP2pTestTweaks(settingsC, peerNameC, passphrase);
const host = startCliInBackground(vaultA, "--settings", settingsA, "p2p-host");
try {
await host.waitUntilContains("P2P host is running", 20000);
const peerFromB = await discoverPeer(vaultB, settingsB, peersTimeout);
const peerFromC = await discoverPeer(vaultC, settingsC, peersTimeout);
const peerFromB = await discoverPeer(vaultB, settingsB, peersTimeout, hostPeerName);
const peerFromC = await discoverPeer(vaultC, settingsC, peersTimeout, hostPeerName);
const targetPath = "p2p/conflicted-from-two-clients.txt";
await runCliWithInputOrFail("from-client-b-v1\n", vaultB, "--settings", settingsB, "put", targetPath);

142
src/common/reportTool.ts Normal file
View File

@@ -0,0 +1,142 @@
import { REMOTE_COUCHDB, REMOTE_MINIO } from "@lib/common/models/setting.const";
import type { ObsidianLiveSyncSettings } from "@lib/common/models/setting.type";
import { generateCredentialObject } from "@lib/replication/httplib";
import { parseHeaderValues } from "@lib/common/utils";
import { requestToCouchDBWithCredentials } from "./utils";
import { LOG_LEVEL_VERBOSE, Logger } from "@lib/common/logger";
import { DEFAULT_SETTINGS } from "@lib/common/models/setting.const.defaults";
import { isCloudantURI } from "@lib/pouchdb/utils_couchdb";
import { compatGlobal } from "@lib/common/coreEnvFunctions";
import { manifestVersion, packageVersion } from "@lib/common/coreEnvVars";
import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore";
function redactObject(obj: Record<string, any>, dotted: string, redactedValue = "REDACTED") {
const keys = dotted.split(".");
let current = obj;
for (let i = 0; i < keys.length - 1; i++) {
const key = keys[i];
if (!(key in current)) {
current[key] = {} as Record<string, any>;
}
// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
current = current[key];
}
const lastKey = keys[keys.length - 1];
if (lastKey in current) {
current[lastKey] = redactedValue;
}
return obj;
}
export async function generateReport(settings: ObsidianLiveSyncSettings, core: LiveSyncBaseCore) {
let responseConfig: Record<string, any> = {};
const REDACTED = "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷";
if (settings.remoteType == REMOTE_COUCHDB) {
try {
const credential = generateCredentialObject(settings);
const customHeaders = parseHeaderValues(settings.couchDB_CustomHeaders);
const r = await requestToCouchDBWithCredentials(
settings.couchDB_URI,
credential,
window.origin,
undefined,
undefined,
undefined,
customHeaders
);
responseConfig = r.json as Record<string, any>;
redactObject(responseConfig, "couch_httpd_auth.secret");
redactObject(responseConfig, "couch_httpd_auth.authentication_db");
redactObject(responseConfig, "couch_httpd_auth.authentication_redirect");
redactObject(responseConfig, "couchdb.uuid");
redactObject(responseConfig, "admins");
redactObject(responseConfig, "users");
redactObject(responseConfig, "chttpd_auth.secret");
delete responseConfig["jwt_keys"];
} catch (ex) {
Logger(ex, LOG_LEVEL_VERBOSE);
responseConfig = {
error: "Requesting information from the remote CouchDB has failed. If you are using IBM Cloudant, this is normal behaviour.",
};
}
} else if (settings.remoteType == REMOTE_MINIO) {
responseConfig = { error: "Object Storage Synchronisation" };
//
}
const defaultKeys = Object.keys(DEFAULT_SETTINGS) as (keyof ObsidianLiveSyncSettings)[];
const pluginConfig = JSON.parse(JSON.stringify(settings)) as ObsidianLiveSyncSettings;
const pluginKeys = Object.keys(pluginConfig);
for (const key of pluginKeys) {
if (defaultKeys.includes(key as keyof ObsidianLiveSyncSettings)) continue;
delete pluginConfig[key as keyof ObsidianLiveSyncSettings];
}
pluginConfig.couchDB_DBNAME = REDACTED;
pluginConfig.couchDB_PASSWORD = REDACTED;
const scheme = pluginConfig.couchDB_URI.startsWith("http:")
? "(HTTP)"
: pluginConfig.couchDB_URI.startsWith("https:")
? "(HTTPS)"
: "";
pluginConfig.couchDB_URI = isCloudantURI(pluginConfig.couchDB_URI) ? "cloudant" : `self-hosted${scheme}`;
pluginConfig.couchDB_USER = REDACTED;
pluginConfig.passphrase = REDACTED;
pluginConfig.encryptedPassphrase = REDACTED;
pluginConfig.encryptedCouchDBConnection = REDACTED;
pluginConfig.accessKey = REDACTED;
pluginConfig.secretKey = REDACTED;
const redact = (source: string) => `${REDACTED}(${source.length} letters)`;
const toSchemeOnly = (uri: string) => {
try {
return `${new URL(uri).protocol}//`;
} catch {
const matched = uri.match(/^[A-Za-z][A-Za-z0-9+.-]*:\/\//);
return matched?.[0] ?? REDACTED;
}
};
pluginConfig.remoteConfigurations = Object.fromEntries(
Object.entries(pluginConfig.remoteConfigurations || {}).map(([id, config]) => [
id,
{
...config,
uri: toSchemeOnly(config.uri),
},
])
);
pluginConfig.region = redact(pluginConfig.region);
pluginConfig.bucket = redact(pluginConfig.bucket);
pluginConfig.pluginSyncExtendedSetting = {};
pluginConfig.P2P_AppID = redact(pluginConfig.P2P_AppID);
pluginConfig.P2P_passphrase = redact(pluginConfig.P2P_passphrase);
pluginConfig.P2P_roomID = redact(pluginConfig.P2P_roomID);
pluginConfig.P2P_relays = redact(pluginConfig.P2P_relays);
pluginConfig.jwtKey = redact(pluginConfig.jwtKey);
pluginConfig.jwtSub = redact(pluginConfig.jwtSub);
pluginConfig.jwtKid = redact(pluginConfig.jwtKid);
pluginConfig.bucketCustomHeaders = redact(pluginConfig.bucketCustomHeaders);
pluginConfig.couchDB_CustomHeaders = redact(pluginConfig.couchDB_CustomHeaders);
pluginConfig.P2P_turnCredential = redact(pluginConfig.P2P_turnCredential);
pluginConfig.P2P_turnUsername = redact(pluginConfig.P2P_turnUsername);
pluginConfig.P2P_turnServers = `(${pluginConfig.P2P_turnServers.split(",").length} servers configured)`;
const endpoint = pluginConfig.endpoint;
if (endpoint == "") {
pluginConfig.endpoint = "Not configured or AWS";
} else {
const endpointScheme = pluginConfig.endpoint.startsWith("http:")
? "(HTTP)"
: pluginConfig.endpoint.startsWith("https:")
? "(HTTPS)"
: "";
pluginConfig.endpoint = `${endpoint.indexOf(".r2.cloudflarestorage.") !== -1 ? "R2" : "self-hosted?"}(${endpointScheme})`;
}
const obsidianInfo = {
navigator: compatGlobal.navigator.userAgent,
fileSystem: core.services.vault.isStorageInsensitive() ? "insensitive" : "sensitive",
};
const result = {
obsidianInfo,
responseConfig,
pluginConfig,
manifestVersion,
packageVersion,
};
return result;
}

View File

@@ -16,10 +16,7 @@
import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore";
import { ConnectionStringParser } from "@lib/common/ConnectionString";
import type { P2PSyncSetting, RemoteConfiguration } from "@lib/common/models/setting.type";
import {
activateP2PRemoteConfiguration,
createRemoteConfigurationId,
} from "@lib/serviceFeatures/remoteConfig";
import { activateP2PRemoteConfiguration, createRemoteConfigurationId } from "@lib/serviceFeatures/remoteConfig";
import { extractP2PRoomSuffix } from "@lib/common/utils";
import { SetupManager } from "@/modules/features/SetupManager";
import SetupRemoteP2P from "@/modules/features/SetupWizard/dialogs/SetupRemoteP2P.svelte";
@@ -36,9 +33,7 @@
let replicatingPeerId = $state<string | null>(null);
let communicatingUntil = $state<Record<string, number>>({});
const COMMUNICATION_HOLD_MS = 2500;
let syncOnReplicationSetting = $state(
core.services.setting.currentSettings()?.P2P_SyncOnReplication ?? ""
);
let syncOnReplicationSetting = $state(core.services.setting.currentSettings()?.P2P_SyncOnReplication ?? "");
type P2PRemoteOption = {
id: string;
name: string;
@@ -51,12 +46,19 @@
let selectingP2PRemote = $state(false);
function addToList(item: string, list: string): string {
const items = list.split(",").map((e) => e.trim()).filter((e) => e);
const items = list
.split(",")
.map((e) => e.trim())
.filter((e) => e);
if (!items.includes(item)) items.push(item);
return items.join(",");
}
function removeFromList(item: string, list: string): string {
return list.split(",").map((e) => e.trim()).filter((e) => e && e !== item).join(",");
return list
.split(",")
.map((e) => e.trim())
.filter((e) => e && e !== item)
.join(",");
}
function markCommunicating(peerId: string) {
@@ -409,7 +411,12 @@
</option>
{/each}
</select>
<button class="icon-button" onclick={() => createAndSelectP2PRemote()} title="Create P2P remote" aria-label="Create P2P remote">
<button
class="icon-button"
onclick={() => createAndSelectP2PRemote()}
title="Create P2P remote"
aria-label="Create P2P remote"
>
+
</button>
</div>
@@ -442,7 +449,8 @@
<div class="peer-item">
<div class="peer-info">
<div class="peer-name">
{peer.name} : <span class="peer-id-mini" title={peer.peerId}>({peer.peerId.slice(0, 8)})</span>
{peer.name} :
<span class="peer-id-mini" title={peer.peerId}>({peer.peerId.slice(0, 8)})</span>
{#if isCommunicating(peer.peerId)}
<span class="comm-icon" title="Communicating" aria-label="Communicating">📡</span>
{/if}
@@ -460,11 +468,11 @@
<button
class="emoji-button"
disabled={replicatingPeerId !== null}
title={replicatingPeerId === peer.peerId ? 'Replicating...' : 'Replicate now'}
aria-label={replicatingPeerId === peer.peerId ? 'Replicating' : 'Replicate now'}
title={replicatingPeerId === peer.peerId ? "Replicating..." : "Replicate now"}
aria-label={replicatingPeerId === peer.peerId ? "Replicating" : "Replicate now"}
onclick={() => startReplication(peer)}
>
{replicatingPeerId === peer.peerId ? '⏳' : '🔄'}
{replicatingPeerId === peer.peerId ? "⏳" : "🔄"}
</button>
<button
class="action-button"
@@ -478,25 +486,31 @@
<span class="decision-label">WATCH</span>
<button
class="emoji-button {isWatching(peer.peerId) ? 'is-watching' : ''}"
title={isWatching(peer.peerId) ? 'Watching this peer \u2014 click to stop' : 'Watch this peer\'s changes'}
aria-label={isWatching(peer.peerId) ? 'Stop watching' : 'Watch peer'}
title={isWatching(peer.peerId)
? "Watching this peer \u2014 click to stop"
: "Watch this peer's changes"}
aria-label={isWatching(peer.peerId) ? "Stop watching" : "Watch peer"}
disabled={!canEditP2PSettings()}
onclick={() => toggleWatch(peer.peerId)}
>
{isWatching(peer.peerId) ? '🔔' : '🔕'}
{isWatching(peer.peerId) ? "🔔" : "🔕"}
</button>
</div> <div class="decision-row watch-row">
</div>
<div class="decision-row watch-row">
<span class="decision-label">SYNC</span>
<button
class="emoji-button {isSyncTarget(peer.name) ? 'is-watching' : ''}"
title={isSyncTarget(peer.name) ? 'Sync target \u2014 click to remove' : 'Set as sync target'}
aria-label={isSyncTarget(peer.name) ? 'Remove sync target' : 'Set sync target'}
title={isSyncTarget(peer.name)
? "Sync target \u2014 click to remove"
: "Set as sync target"}
aria-label={isSyncTarget(peer.name) ? "Remove sync target" : "Set sync target"}
disabled={!canEditP2PSettings()}
onclick={() => toggleSyncTarget(peer)}
>
{isSyncTarget(peer.name) ? '🔗' : '⛓️‍💥'}
{isSyncTarget(peer.name) ? "🔗" : "⛓️‍💥"}
</button>
</div> {:else}
</div>
{:else}
<div class="decision-status">
<span class="badge status-chip {getAcceptanceStatusClass(peer)}">
{getAcceptanceStatus(peer)}
@@ -571,7 +585,6 @@
display: flex;
flex-direction: column;
gap: 1rem;
padding: 0.75rem;
}
.peers-section {
@@ -584,7 +597,7 @@
display: flex;
justify-content: space-between;
align-items: center;
flex-wrap: nowrap;
flex-wrap: wrap;
gap: 0.5rem;
}
@@ -603,8 +616,9 @@
}
.remote-picker {
max-width: 14rem;
min-width: 8rem;
max-width: 10rem;
min-width: 1em;
flex-shrink: 1;
height: 1.9rem;
border: 1px solid var(--divider-color);
border-radius: 0.4rem;
@@ -648,6 +662,7 @@
.peers-header {
display: flex;
flex-wrap: wrap;
justify-content: space-between;
align-items: center;
gap: 0.5rem;
@@ -873,5 +888,4 @@
font-size: 0.9rem;
padding: 1rem;
}
</style>
</style>

Submodule src/lib updated: 36b99354f6...b9aaf3c03a

View File

@@ -2,9 +2,11 @@ import { Logger, LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
import { extractObject } from "octagonal-wheels/object";
import {
TweakValuesShouldMatchedTemplate,
TweakValuesTemplate,
IncompatibleChanges,
confName,
type TweakValues,
type ObsidianLiveSyncSettings,
type RemoteDBSettings,
IncompatibleChangesInSpecificPattern,
CompatibleButLossyChanges,
@@ -16,7 +18,105 @@ import type { InjectableServiceHub } from "../../lib/src/services/InjectableServ
import type { LiveSyncCore } from "../../main.ts";
import { REMOTE_P2P } from "@lib/common/models/setting.const.ts";
function valueToString(value: any) {
if (typeof value === "boolean") {
return value ? "true" : "false";
}
if (typeof value === "object") {
return JSON.stringify(value);
}
return `${value}`;
}
export class ModuleResolvingMismatchedTweaks extends AbstractModule {
private _hasNotifiedAutoAcceptCompatibleUndefined = false;
private _collectMismatchedTweakKeys(current: TweakValues, preferred: Partial<TweakValues>) {
const items = Object.keys(
TweakValuesShouldMatchedTemplate
) as (keyof typeof TweakValuesShouldMatchedTemplate)[];
return items.filter((key) => current[key] !== preferred[key]);
}
private _selectNewerTweakSide(current: TweakValues, preferred: Partial<TweakValues>): "REMOTE" | "CURRENT" {
Logger(`Modified: ${current.tweakModified} (current) vs ${preferred.tweakModified} (preferred)`);
const currentModified = current.tweakModified;
const preferredModified = preferred.tweakModified;
// debugger;
const hasCurrentModified = typeof currentModified === "number" && currentModified > 0;
const hasPreferredModified = typeof preferredModified === "number" && preferredModified > 0;
if (!hasCurrentModified && !hasPreferredModified) return "REMOTE";
if (!hasCurrentModified) return "REMOTE";
if (!hasPreferredModified) return "CURRENT";
if (preferredModified >= currentModified) return "REMOTE";
return "CURRENT";
}
private async _shouldAutoAcceptCompatibleLossy(
current: TweakValues,
preferred: Partial<TweakValues>,
mismatchedKeys: (keyof typeof TweakValuesShouldMatchedTemplate)[]
): Promise<"REMOTE" | "CURRENT" | undefined> {
if (mismatchedKeys.length === 0) return undefined;
const hasOnlyCompatibleLossyMismatches = mismatchedKeys.every(
(key) => CompatibleButLossyChanges.indexOf(key) !== -1
);
if (!hasOnlyCompatibleLossyMismatches) return undefined;
if (this.settings.autoAcceptCompatibleTweak === undefined) {
if (this._hasNotifiedAutoAcceptCompatibleUndefined) {
return undefined;
}
this._hasNotifiedAutoAcceptCompatibleUndefined = true;
const CHOICE_ENABLE = $msg("TweakMismatchResolve.Action.EnableAutoAcceptCompatible");
const CHOICE_DISABLE = $msg("TweakMismatchResolve.Action.DisableAutoAcceptCompatible");
const CHOICES = [CHOICE_ENABLE, CHOICE_DISABLE] as const;
const message = $msg("TweakMismatchResolve.Message.AutoAcceptCompatibleUndefined");
const ret = await this.core.confirm.askSelectStringDialogue(message, CHOICES, {
title: $msg("TweakMismatchResolve.Title.AutoAcceptCompatible"),
timeout: 0,
defaultAction: CHOICE_ENABLE,
});
if (ret !== CHOICE_ENABLE) {
return undefined;
}
await this.services.setting.applyPartial(
{
autoAcceptCompatibleTweak: true,
},
true
);
Logger("Auto-accept for compatible tweak mismatch has been enabled.");
}
if (this.settings.autoAcceptCompatibleTweak !== true) return undefined;
return this._selectNewerTweakSide(current, preferred);
}
/**
* Hook before saving settings, to check if there are changes in tweak values, and if so,
* update the tweakModified timestamp to current time.
* This allows other devices to know that the tweak values have been changed and decide whether to accept the new values based on the modification time.
* @param next
* @param previous
* @returns
*/
async _onBeforeSaveSettingData(next: ObsidianLiveSyncSettings, previous: ObsidianLiveSyncSettings) {
const tweakKeys = Object.keys(TweakValuesTemplate) as (keyof TweakValues)[];
const tweakKeysForUpdate = tweakKeys.filter((key) => key !== "tweakModified");
const hasChangedTweak = tweakKeysForUpdate.some((key) => next[key] !== previous[key]);
if (!hasChangedTweak) return;
Logger(
`Some tweak values have been changed. ${tweakKeysForUpdate.filter((key) => next[key] !== previous[key]).join(", ")}`
);
const modified = Date.now();
Logger(`Modified: ${modified}`);
return await Promise.resolve({
tweakModified: modified,
});
}
async _anyAfterConnectCheckFailed(): Promise<boolean | "CHECKAGAIN" | undefined> {
if (!this.core.replicator.tweakSettingsMismatched && !this.core.replicator.preferredTweakValue) return false;
const preferred = this.core.replicator.preferredTweakValue;
@@ -27,10 +127,16 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
if (ret == "IGNORE") return true;
}
async _checkAndAskResolvingMismatchedTweaks(
preferred: Partial<TweakValues>
): Promise<[TweakValues | boolean, boolean]> {
const mine = extractObject(TweakValuesShouldMatchedTemplate, this.settings);
async _checkAndAskResolvingMismatchedTweaks(preferred: TweakValues): Promise<[TweakValues | boolean, boolean]> {
const mine = extractObject(TweakValuesTemplate, this.settings) as TweakValues;
const mismatchedKeys = this._collectMismatchedTweakKeys(mine, preferred);
const autoAcceptSide = await this._shouldAutoAcceptCompatibleLossy(mine, preferred, mismatchedKeys);
if (autoAcceptSide === "REMOTE") {
return [{ ...mine, ...preferred }, false];
}
if (autoAcceptSide === "CURRENT") {
return [true, false];
}
const items = Object.entries(TweakValuesShouldMatchedTemplate);
let rebuildRequired = false;
let rebuildRecommended = false;
@@ -69,8 +175,8 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
tableRows.push(
$msg("TweakMismatchResolve.Table.Row", {
name: confName(key),
self: valueMine,
remote: valuePreferred,
self: valueToString(valueMine),
remote: valueToString(valuePreferred),
})
);
}
@@ -137,9 +243,7 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
if (!tweaks) {
return "IGNORE";
}
const preferred = extractObject(TweakValuesShouldMatchedTemplate, tweaks);
const [conf, rebuildRequired] = await this.services.tweakValue.checkAndAskResolvingMismatched(preferred);
const [conf, rebuildRequired] = await this.services.tweakValue.checkAndAskResolvingMismatched(tweaks);
if (!conf) return "IGNORE";
if (conf === true) {
@@ -147,10 +251,7 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
if (rebuildRequired) {
await this.core.rebuilder.$rebuildRemote();
}
Logger(
`Tweak values on the remote server have been updated. Your other device will see this message.`,
LOG_LEVEL_NOTICE
);
Logger($msg("TweakMismatchResolve.Message.remoteUpdated"), LOG_LEVEL_NOTICE);
return "CHECKAGAIN";
}
if (conf) {
@@ -160,7 +261,7 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
if (rebuildRequired) {
await this.core.rebuilder.$fetchLocal();
}
Logger(`Configuration has been updated as configured by the other device.`, LOG_LEVEL_NOTICE);
Logger($msg("TweakMismatchResolve.Message.mineUpdated"), LOG_LEVEL_NOTICE);
return "CHECKAGAIN";
}
return "IGNORE";
@@ -201,6 +302,16 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
trialSetting: RemoteDBSettings,
preferred: TweakValues
): Promise<{ result: false | TweakValues; requireFetch: boolean }> {
const localTweaks = extractObject(TweakValuesTemplate, this.settings) as TweakValues;
const mismatchedKeys = this._collectMismatchedTweakKeys(localTweaks, preferred);
const autoAcceptSide = await this._shouldAutoAcceptCompatibleLossy(localTweaks, preferred, mismatchedKeys);
if (autoAcceptSide === "REMOTE") {
return { result: { ...trialSetting, ...preferred }, requireFetch: false };
}
if (autoAcceptSide === "CURRENT") {
return { result: false, requireFetch: false };
}
const items = Object.entries(TweakValuesShouldMatchedTemplate);
let rebuildRequired = false;
let rebuildRecommended = false;
@@ -211,8 +322,8 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
// const items = [mine,preferred]
for (const v of items) {
const key = v[0] as keyof typeof TweakValuesShouldMatchedTemplate;
const remoteValueForDisplay = escapeMarkdownValue(preferred[key]);
const currentValueForDisplay = `${escapeMarkdownValue((trialSetting as TweakValues)?.[key])}`;
const remoteValueForDisplay = escapeMarkdownValue(valueToString(preferred[key]));
const currentValueForDisplay = escapeMarkdownValue(valueToString((trialSetting as TweakValues)?.[key]));
if ((trialSetting as TweakValues)?.[key] !== preferred[key]) {
if (IncompatibleChanges.indexOf(key) !== -1) {
rebuildRequired = true;
@@ -289,6 +400,7 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
}
override onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
services.setting.onBeforeSaveSettingData.addHandler(this._onBeforeSaveSettingData.bind(this));
services.tweakValue.fetchRemotePreferred.setHandler(this._fetchRemotePreferredTweakValues.bind(this));
services.tweakValue.checkAndAskResolvingMismatched.setHandler(
this._checkAndAskResolvingMismatchedTweaks.bind(this)

View File

@@ -0,0 +1,108 @@
import { describe, expect, it, vi } from "vitest";
import { DEFAULT_SETTINGS, REMOTE_COUCHDB, type RemoteDBSettings, type TweakValues } from "@lib/common/types";
import { ModuleResolvingMismatchedTweaks } from "./ModuleResolveMismatchedTweaks";
function createModule(settingsOverride: Partial<typeof DEFAULT_SETTINGS> = {}) {
const askSelectStringDialogue = vi.fn(async () => undefined);
const core = {
_services: {
API: {
addLog: vi.fn(),
addCommand: vi.fn(),
registerWindow: vi.fn(),
addRibbonIcon: vi.fn(),
registerProtocolHandler: vi.fn(),
},
setting: {
saveSettingData: vi.fn(async () => undefined),
},
},
settings: {
...DEFAULT_SETTINGS,
remoteType: REMOTE_COUCHDB,
...settingsOverride,
},
confirm: {
askSelectStringDialogue,
},
} as any;
Object.defineProperty(core, "services", {
get() {
return core._services;
},
});
const module = new ModuleResolvingMismatchedTweaks(core);
return { module, core, askSelectStringDialogue };
}
describe("ModuleResolvingMismatchedTweaks", () => {
it("should auto-accept compatible mismatches on connect check using newer remote tweakModified", async () => {
const { module, askSelectStringDialogue } = createModule({
autoAcceptCompatibleTweak: true,
hashAlg: "xxhash64",
tweakModified: 100,
});
const preferred = {
...(DEFAULT_SETTINGS as unknown as TweakValues),
hashAlg: "xxhash32",
tweakModified: 200,
} as Partial<TweakValues>;
const [conf, rebuild] = await module._checkAndAskResolvingMismatchedTweaks(preferred);
expect(conf).toEqual(preferred);
expect(rebuild).toBe(false);
expect(askSelectStringDialogue).not.toHaveBeenCalled();
});
it("should fallback to manual confirmation when mismatches are mixed on connect check", async () => {
const { module, askSelectStringDialogue } = createModule({
autoAcceptCompatibleTweak: true,
hashAlg: "xxhash64",
encrypt: false,
tweakModified: 100,
});
const preferred = {
...(DEFAULT_SETTINGS as unknown as TweakValues),
hashAlg: "xxhash32",
encrypt: true,
tweakModified: 200,
} as Partial<TweakValues>;
const [conf, rebuild] = await module._checkAndAskResolvingMismatchedTweaks(preferred);
expect(conf).toBe(false);
expect(rebuild).toBe(false);
expect(askSelectStringDialogue).toHaveBeenCalledTimes(1);
});
it("should auto-accept compatible mismatches on remote-config check using newer local tweakModified", async () => {
const { module, askSelectStringDialogue } = createModule({
autoAcceptCompatibleTweak: true,
hashAlg: "xxhash64",
tweakModified: 300,
});
const trialSetting = {
...DEFAULT_SETTINGS,
remoteType: REMOTE_COUCHDB,
hashAlg: "xxhash64",
tweakModified: 300,
} as RemoteDBSettings;
const preferred = {
...(trialSetting as unknown as TweakValues),
hashAlg: "xxhash32",
tweakModified: 200,
} as TweakValues;
const result = await module._askUseRemoteConfiguration(trialSetting, preferred);
expect(result).toEqual({ result: false, requireFetch: false });
expect(askSelectStringDialogue).not.toHaveBeenCalled();
});
});

View File

@@ -121,7 +121,7 @@ export class ModuleObsidianEvents extends AbstractObsidianModule {
return;
}
const isHidden = document.hidden;
const isHidden = activeWindow.document.hidden;
if (this.isLastHidden === isHidden) {
return;
}
@@ -134,7 +134,7 @@ export class ModuleObsidianEvents extends AbstractObsidianModule {
} else {
// suspend all temporary.
if (this.services.appLifecycle.isSuspended()) return;
if (!this.hasFocus) return;
// Do not block resume by focus state here; visibility recovery should be enough.
await this.services.appLifecycle.onResuming();
await this.services.appLifecycle.onResumed();
}

View File

@@ -25,7 +25,7 @@ import {
EVENT_ON_UNRESOLVED_ERROR,
} from "../../common/events.ts";
import { AbstractObsidianModule } from "../AbstractObsidianModule.ts";
import { addIcon, normalizePath, Notice } from "../../deps.ts";
import { addIcon, debounce, normalizePath, Notice, stringifyYaml, type WorkspaceLeaf } from "../../deps.ts";
import { LOG_LEVEL_NOTICE, setGlobalLogFunction } from "octagonal-wheels/common/logger";
import { LogPaneView, VIEW_TYPE_LOG } from "./Log/LogPaneView.ts";
import { serialized } from "octagonal-wheels/concurrency/lock";
@@ -41,6 +41,8 @@ import {
} from "@lib/string_and_binary/path.ts";
import { MARK_LOG_NETWORK_ERROR, MARK_LOG_SEPARATOR } from "@lib/services/lib/logUtils.ts";
import { NetworkWarningStyles } from "@lib/common/models/setting.const.ts";
import { compatGlobal } from "@lib/common/coreEnvFunctions.ts";
import { generateReport } from "@/common/reportTool.ts";
// This module cannot be a core module because it depends on the Obsidian UI.
@@ -50,18 +52,51 @@ const globalLogFunction = (message: any, level?: number, key?: string) => {
const messageX =
message instanceof Error
? new LiveSyncError("[Error Logged]: " + message.message, { cause: message })
: message;
: typeof message === "string"
? message
: JSON.stringify(message);
const entry = { message: messageX, level, key } as LogEntry;
recentLogEntries.value = [...recentLogEntries.value, entry];
};
setGlobalLogFunction(globalLogFunction);
let recentLogs = [] as string[];
// Keep the recent logs in memory for display, but also keep a longer history in logForDump for when the user wants to see more logs.
// logForDump is not reactive and is only used for dumping logs when requested, while recentLogs is reactive and is used for displaying logs in the UI.
const logForDump = [] as string[];
function addLog(log: string) {
recentLogs = [...recentLogs, log].splice(-200);
logMessages.value = recentLogs;
logForDump.push(log);
while (logForDump.length > 1000) {
logForDump.shift();
}
}
// Display log is kept separate from the full log history to optimize performance and memory usage.
// And debounce the updates to the display log to avoid excessive UI updates when there are many log entries in a short time.
const logForDisplay = [] as string[];
const updateLogMessage = debounce(() => {
logMessages.value = [...logForDisplay];
}, 25);
function addDisplayLog(log: string) {
logForDisplay.push(log);
while (logForDisplay.length > 200) {
logForDisplay.shift();
}
updateLogMessage();
}
const redactPatterns = [/PBKDF2 salt \(Security Seed\):.*$/];
function redactLog(log: string) {
let redactedLog = log;
for (const pattern of redactPatterns) {
redactedLog = redactedLog.replace(pattern, (match) => {
return match.split(":")[0] + ": [REDACTED]";
});
}
return redactedLog;
}
// logStore.intercept(e => e.slice(Math.min(e.length - 200, 0)));
const showDebugLog = false;
@@ -86,15 +121,15 @@ export class ModuleLog extends AbstractObsidianModule {
// const emptyMark = `\u{2003}`;
function padLeftSpComputed(numI: ReactiveValue<number>, mark: string) {
const formatted = reactiveSource("");
let timer: ReturnType<typeof setTimeout> | undefined = undefined;
let timer: number | undefined = undefined;
let maxLen = 1;
numI.onChanged((numX) => {
const num = numX.value;
const numLen = `${Math.abs(num)}`.length + 1;
maxLen = maxLen < numLen ? numLen : maxLen;
if (timer) clearTimeout(timer);
if (timer) compatGlobal.clearTimeout(timer);
if (num == 0) {
timer = setTimeout(() => {
timer = compatGlobal.setTimeout(() => {
formatted.value = "";
maxLen = 1;
}, 3000);
@@ -323,7 +358,7 @@ export class ModuleLog extends AbstractObsidianModule {
if (this.nextFrameQueue) {
return;
}
this.nextFrameQueue = requestAnimationFrame(() => {
this.nextFrameQueue = compatGlobal.requestAnimationFrame(() => {
this.nextFrameQueue = undefined;
const { message, status } = this.statusBarLabels.value;
// const recent = logMessages.value;
@@ -346,7 +381,8 @@ export class ModuleLog extends AbstractObsidianModule {
(a, b) => (a < b.ttl ? a : b.ttl),
Number.MAX_SAFE_INTEGER
);
if (this.logLines.length > 0) setTimeout(() => this.applyStatusBarText(), minimumNext - now);
if (this.logLines.length > 0)
compatGlobal.setTimeout(() => this.applyStatusBarText(), minimumNext - now);
const recent = this.logLines.map((e) => e.message);
const recentLogs = recent.reverse().join("\n");
if (isDirty("recentLogs", recentLogs)) this.logHistory!.innerText = recentLogs;
@@ -368,7 +404,7 @@ export class ModuleLog extends AbstractObsidianModule {
if (this.statusDiv) {
this.statusDiv.remove();
}
document.querySelectorAll(`.livesync-status`)?.forEach((e) => e.remove());
compatGlobal.document.querySelectorAll(`.livesync-status`)?.forEach((e) => e.remove());
return Promise.resolve(true);
}
_everyOnloadStart(): Promise<boolean> {
@@ -390,7 +426,28 @@ export class ModuleLog extends AbstractObsidianModule {
void this.services.API.showWindow(VIEW_TYPE_LOG);
},
});
this.registerView(VIEW_TYPE_LOG, (leaf) => new LogPaneView(leaf, this.plugin));
this.addCommand({
id: "dump-debug-info",
name: "Generate full report for opening the issue with debug info",
callback: async () => {
const recentLog = [...logForDump];
const report = await generateReport(this.services.setting.currentSettings(), this.core);
const info = {
...report,
recentLog: recentLog.map(redactLog),
};
const yaml = `\`\`\`\`
# ---- Debug Info Dump ----
${stringifyYaml(info)}
\`\`\`\``;
if (await this.services.UI.promptCopyToClipboard("Debug info", yaml)) {
new Notice(
"Debug info copied to clipboard. You can paste it in the issue. Be careful as it may contain sensitive information, review it before sharing."
);
}
},
});
this.registerView(VIEW_TYPE_LOG, (leaf: WorkspaceLeaf) => new LogPaneView(leaf, this.plugin));
return Promise.resolve(true);
}
private _everyOnloadAfterLoadSettings(): Promise<boolean> {
@@ -404,7 +461,7 @@ export class ModuleLog extends AbstractObsidianModule {
void this.setFileStatus();
});
const w = document.querySelectorAll(`.livesync-status`);
const w = compatGlobal.document.querySelectorAll(`.livesync-status`);
w.forEach((e) => e.remove());
this.observeForLogs();
@@ -421,6 +478,8 @@ export class ModuleLog extends AbstractObsidianModule {
this.statusBar?.addClass("syncstatusbar");
}
this.adjustStatusDivPosition();
this._log("Log module loaded", LOG_LEVEL_INFO);
this._log("Verbose log", LOG_LEVEL_VERBOSE);
return Promise.resolve(true);
}
@@ -444,11 +503,12 @@ export class ModuleLog extends AbstractObsidianModule {
if (level == LOG_LEVEL_DEBUG && !showDebugLog) {
return;
}
let memoOnly = false;
if (level <= LOG_LEVEL_INFO && this.settings && this.settings.lessInformationInLog) {
return;
memoOnly = true;
}
if (this.settings && !this.settings.showVerboseLog && level == LOG_LEVEL_VERBOSE) {
return;
memoOnly = true;
}
const vaultName = this.services.vault.getVaultName();
const now = new Date();
@@ -469,6 +529,15 @@ export class ModuleLog extends AbstractObsidianModule {
? `${errorInfo}`
: JSON.stringify(message, null, 2);
const newMessage = timestamp + "->" + messageContent;
if (this.settings?.writeLogToTheFile) {
this.writeLogToTheFile(now, vaultName, newMessage);
}
addLog(newMessage);
if (memoOnly) {
return;
}
addDisplayLog(newMessage);
if (message instanceof Error) {
console.error(vaultName + ":" + newMessage);
} else if (level >= LOG_LEVEL_INFO) {
@@ -479,10 +548,6 @@ export class ModuleLog extends AbstractObsidianModule {
if (!this.settings?.showOnlyIconsOnEditor) {
this.statusLog.value = messageContent;
}
if (this.settings?.writeLogToTheFile) {
this.writeLogToTheFile(now, vaultName, newMessage);
}
addLog(newMessage);
this.logLines.push({ ttl: now.getTime() + 3000, message: newMessage });
if (level >= LOG_LEVEL_NOTICE) {

View File

@@ -35,6 +35,7 @@ export function paneAdvanced(this: ObsidianLiveSyncSettingTab, paneEl: HTMLEleme
clampMin: 10,
onUpdate: this.onlyOnCouchDB,
});
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("autoAcceptCompatibleTweak");
// new Setting(paneEl)
// .setClass("wizardHidden")
// .autoWireToggle("sendChunksBulk", { onUpdate: onlyOnCouchDB })

View File

@@ -39,6 +39,7 @@ import { EVENT_REQUEST_SHOW_HISTORY } from "../../../common/obsidianEvents.ts";
import { generateCredentialObject } from "../../../lib/src/replication/httplib.ts";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import type { PageFunctions } from "./SettingPane.ts";
import { generateReport } from "@/common/reportTool.ts";
export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement, { addPanel }: PageFunctions): void {
// const hatchWarn = this.createEl(paneEl, "div", { text: `To stop the boot up sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` });
// hatchWarn.addClass("op-warn-info");
@@ -69,140 +70,14 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
eventHub.emitEvent(EVENT_REQUEST_RUN_FIX_INCOMPLETE);
})
);
new Setting(paneEl).setName($msg("Prepare the 'report' to create an issue")).addButton((button) =>
button
.setButtonText($msg("Copy Report to clipboard"))
.setCta()
.setDisabled(false)
.onClick(async () => {
let responseConfig: any = {};
const REDACTED = "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷";
if (this.editingSettings.remoteType == REMOTE_COUCHDB) {
try {
const credential = generateCredentialObject(this.editingSettings);
const customHeaders = parseHeaderValues(this.editingSettings.couchDB_CustomHeaders);
const r = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI,
credential,
window.origin,
undefined,
undefined,
undefined,
customHeaders
);
Logger(JSON.stringify(r.json, null, 2));
responseConfig = r.json;
responseConfig["couch_httpd_auth"].secret = REDACTED;
responseConfig["couch_httpd_auth"].authentication_db = REDACTED;
responseConfig["couch_httpd_auth"].authentication_redirect = REDACTED;
responseConfig["couchdb"].uuid = REDACTED;
responseConfig["admins"] = REDACTED;
delete responseConfig["jwt_keys"];
if ("secret" in responseConfig["chttpd_auth"])
responseConfig["chttpd_auth"].secret = REDACTED;
} catch (ex) {
Logger(ex, LOG_LEVEL_VERBOSE);
responseConfig = {
error: "Requesting information from the remote CouchDB has failed. If you are using IBM Cloudant, this is normal behaviour.",
};
}
} else if (this.editingSettings.remoteType == REMOTE_MINIO) {
responseConfig = { error: "Object Storage Synchronisation" };
//
}
const defaultKeys = Object.keys(DEFAULT_SETTINGS) as (keyof ObsidianLiveSyncSettings)[];
const pluginConfig = JSON.parse(JSON.stringify(this.editingSettings)) as ObsidianLiveSyncSettings;
const pluginKeys = Object.keys(pluginConfig);
for (const key of pluginKeys) {
if (defaultKeys.includes(key as any)) continue;
delete pluginConfig[key as keyof ObsidianLiveSyncSettings];
}
pluginConfig.couchDB_DBNAME = REDACTED;
pluginConfig.couchDB_PASSWORD = REDACTED;
const scheme = pluginConfig.couchDB_URI.startsWith("http:")
? "(HTTP)"
: pluginConfig.couchDB_URI.startsWith("https:")
? "(HTTPS)"
: "";
pluginConfig.couchDB_URI = isCloudantURI(pluginConfig.couchDB_URI)
? "cloudant"
: `self-hosted${scheme}`;
pluginConfig.couchDB_USER = REDACTED;
pluginConfig.passphrase = REDACTED;
pluginConfig.encryptedPassphrase = REDACTED;
pluginConfig.encryptedCouchDBConnection = REDACTED;
pluginConfig.accessKey = REDACTED;
pluginConfig.secretKey = REDACTED;
const redact = (source: string) => `${REDACTED}(${source.length} letters)`;
const toSchemeOnly = (uri: string) => {
try {
return `${new URL(uri).protocol}//`;
} catch {
const matched = uri.match(/^[A-Za-z][A-Za-z0-9+.-]*:\/\//);
return matched?.[0] ?? REDACTED;
}
};
pluginConfig.remoteConfigurations = Object.fromEntries(
Object.entries(pluginConfig.remoteConfigurations || {}).map(([id, config]) => [
id,
{
...config,
uri: toSchemeOnly(config.uri),
},
])
);
pluginConfig.region = redact(pluginConfig.region);
pluginConfig.bucket = redact(pluginConfig.bucket);
pluginConfig.pluginSyncExtendedSetting = {};
pluginConfig.P2P_AppID = redact(pluginConfig.P2P_AppID);
pluginConfig.P2P_passphrase = redact(pluginConfig.P2P_passphrase);
pluginConfig.P2P_roomID = redact(pluginConfig.P2P_roomID);
pluginConfig.P2P_relays = redact(pluginConfig.P2P_relays);
pluginConfig.jwtKey = redact(pluginConfig.jwtKey);
pluginConfig.jwtSub = redact(pluginConfig.jwtSub);
pluginConfig.jwtKid = redact(pluginConfig.jwtKid);
pluginConfig.bucketCustomHeaders = redact(pluginConfig.bucketCustomHeaders);
pluginConfig.couchDB_CustomHeaders = redact(pluginConfig.couchDB_CustomHeaders);
pluginConfig.P2P_turnCredential = redact(pluginConfig.P2P_turnCredential);
pluginConfig.P2P_turnUsername = redact(pluginConfig.P2P_turnUsername);
pluginConfig.P2P_turnServers = `(${pluginConfig.P2P_turnServers.split(",").length} servers configured)`;
const endpoint = pluginConfig.endpoint;
if (endpoint == "") {
pluginConfig.endpoint = "Not configured or AWS";
} else {
const endpointScheme = pluginConfig.endpoint.startsWith("http:")
? "(HTTP)"
: pluginConfig.endpoint.startsWith("https:")
? "(HTTPS)"
: "";
pluginConfig.endpoint = `${endpoint.indexOf(".r2.cloudflarestorage.") !== -1 ? "R2" : "self-hosted?"}(${endpointScheme})`;
}
const obsidianInfo = {
navigator: navigator.userAgent,
fileSystem: this.core.services.vault.isStorageInsensitive() ? "insensitive" : "sensitive",
};
const msgConfig = `# ---- Obsidian info ----
${stringifyYaml(obsidianInfo)}
---
# ---- remote config ----
${stringifyYaml(responseConfig)}
---
# ---- Plug-in config ----
${stringifyYaml({
version: this.manifestVersion,
...pluginConfig,
})}`;
console.log(msgConfig);
if ((await this.services.UI.promptCopyToClipboard("Generated report", msgConfig)) == true) {
// await navigator.clipboard.writeText(msgConfig);
// Logger(
// `Generated report has been copied to clipboard. Please report the issue with this! Thank you for your cooperation!`,
// LOG_LEVEL_NOTICE
// );
}
await this.app.commands.executeCommandById("obsidian-livesync:dump-debug-info");
})
);
new Setting(paneEl)

View File

@@ -34,11 +34,11 @@ const terserOption = {
inline: false,
join_vars: true,
loops: true,
passes: 4,
passes: 1,
reduce_vars: true,
reduce_funcs: false,
arrows: true,
collapse_vars: true,
collapse_vars: false,
comparisons: true,
//@ts-ignore
lhs_constants: true,

View File

@@ -3,12 +3,45 @@ Since 19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
The head note of 0.25 is now in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md). Because 0.25 got a lot of updates, thankfully, compatibility is kept and we do not need breaking changes! In other words, when get enough stabled. The next version will be v1.0.0. Even though it my hope.
## Unreleased (0.25.64-patch1)
## 0.25.68
18th May, 2026
22nd May, 2026
### Improved
- Improved an error verbosity on concurrent processing on start-up process.
- P2P connections have improved slightly
- Upgrade to `trystero` v0.24.0, and fixes event handler assignment. This should fix some edge cases where P2P connections fail to establish or messages are not properly handled.
- Weaken terser options to avoid potential issues with minification that could cause runtime errors in some environments.
## ~~0.25.66~~ 0.25.67
20th May, 2026
0.25.66 had a bug that the auto-accept logic for compatible but lossy mismatches was not working as intended.
### New features
- Implement an auto-accept compatible tweak setting and enhance the mismatch resolution logic.
### Improved
- Many messages related to tweak mismatch resolution have been updated for clarity.
## 0.25.65
19th May, 2026
### Fixed
- Fix an issue about resuming from background on iOS (#888).
- Now Chunk Splitter: `V3: Fine Deduplication` is working fine again (#866).
- It has some drawbacks, such as fewer chunks are generated. However, it makes less transfer and storage when the files are modified but not completely changed.
- Unsynchronised local changes (which means changes that have not been sent) are now correctly preserved as a conflict (Thank you so much for @SeleiXi!).
- Avoid creating a new revision when the current and conflicted revisions have identical content (Thank you so much for @daichi-629).
### Improved
- Improved the error verbosity on concurrent processing during the start-up process.
- Now the `report` includes recent logs (of verbosity `verbose` even settings is not set to `verbose`).
- Updating logs is now debounced to avoid excessive updates during rapid log generation.
- Added a `Generate full report for opening the issue with debug info` command to the command palette, which generates a report without opening the settings dialogue.
## 0.25.64

View File

@@ -0,0 +1,197 @@
import { glob } from "glob";
import { resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { promises as fs } from "node:fs";
import { isPlainText, shouldSplitAsPlainText } from "../../src/lib/src/string_and_binary/path";
import { splitPiecesRabinKarp } from "../../src/lib/src/string_and_binary/chunks";
import {
PREFERRED_BASE,
PREFERRED_JOURNAL_SYNC,
PREFERRED_SETTING_CLOUDANT,
PREFERRED_SETTING_SELF_HOSTED,
} from "../../src/lib/src/common/models/setting.const.preferred";
import { type ObsidianLiveSyncSettings, DEFAULT_SETTINGS, MAX_DOC_SIZE_BIN } from "../../src/lib/src/common/types";
async function blobFromString(content: string): Promise<Blob> {
return new Blob([content], { type: "text/plain" });
}
const preferred = PREFERRED_BASE;
const preferredJournal = PREFERRED_JOURNAL_SYNC;
const preferredCouchDB = PREFERRED_SETTING_SELF_HOSTED;
const preferredIBM = PREFERRED_SETTING_CLOUDANT;
function computeChunkSize(overlay: Partial<ObsidianLiveSyncSettings>) {
const settings = { ...DEFAULT_SETTINGS, ...overlay };
const maxChunkSize = Math.floor(MAX_DOC_SIZE_BIN * ((settings.customChunkSize || 0) * 1 + 1));
const pieceSize = maxChunkSize;
const minimumChunkSize = settings.minimumChunkSize;
return { pieceSize, minimumChunkSize };
}
async function testSplit(
splitPiecesRabinKarpFn: typeof splitPiecesRabinKarp,
content: Blob,
settingsOverlay: Partial<ObsidianLiveSyncSettings>
) {
const { pieceSize, minimumChunkSize } = computeChunkSize(settingsOverlay);
const isPlain = content.type === "text/plain";
const chunkGenerator = await splitPiecesRabinKarpFn(content, pieceSize, isPlain, minimumChunkSize);
const chunks = [] as string[];
for await (const chunk of chunkGenerator()) {
chunks.push(chunk);
}
// if there are few chunks, calculate average chunk size except the last chunk which can be smaller due to the way the algorithm works, especially for small files.
const averageChunkSize =
chunks.length > 1
? chunks.slice(0, -1).reduce((acc, chunk) => acc + chunk.length, 0) / (chunks.length - 1)
: chunks.reduce((acc, chunk) => acc + chunk.length, 0) / chunks.length;
const lastChunk = chunks[chunks.length - 1];
// compute minimum chunk size if the last chunk is not the smallest.
const nonLastChunkSizes = chunks.slice(0, -1).map((c) => c.length);
const minChunkSize = nonLastChunkSizes.length > 0 ? Math.min(...nonLastChunkSizes) : lastChunk.length;
const result = {
isPlain,
originalSize: content.size,
chunkCount: chunks.length,
totalLength: chunks.reduce((acc, chunk) => acc + chunk.length, 0),
averageChunkSize: averageChunkSize,
maxChunkSize: Math.max(...chunks.map((c) => c.length)),
minChunkSize: minChunkSize,
uniqueChunks: new Set(chunks).size,
chunks: chunks,
};
return result;
}
const __filename = fileURLToPath(import.meta.url);
const __dirname = resolve(__filename, "..");
async function loadFileAsBlob(filePath: string): Promise<Blob> {
if (shouldSplitAsPlainText(filePath)) {
const content = await fs.readFile(filePath, "utf-8");
return blobFromString(content);
} else {
const buffer = await fs.readFile(filePath);
return new Blob([buffer]);
}
}
const testProfiles = [
{ name: "CouchDB", settings: preferredCouchDB },
{ name: "IBM Cloudant", settings: preferredIBM },
{ name: "Journal Sync", settings: preferredJournal },
// { name: "Base", settings: preferred },
];
function modifyBlob(blob: Blob, position: number, insertText: string): Blob {
const before = blob.slice(0, position);
const after = blob.slice(position);
const insert = new Blob([insertText], { type: blob.type });
return new Blob([before, insert, after], { type: blob.type });
}
async function main() {
const results = [] as string[][];
console.log("directory:", __dirname);
const findPath = resolve(__dirname, "../../");
console.warn("CWD:", findPath);
let testFiles = await glob("**/*.*", {
cwd: findPath,
maxDepth: 20,
ignore: ["**/node_modules/**", "**/.obsidian/**", "**/dist/**", "**/build/**", "**/out/**"],
});
testFiles = testFiles.filter((file) => {
const ext = file.split(".").pop()?.toLowerCase() || "";
return ["md", "txt", "json", "csv", "png"].includes(ext);
});
const header = [
"Profile",
"Implementation",
"Edition",
"File",
"Mode",
"Original Size (bytes)",
"Chunk Count",
"Average Chunk Size",
"Max Chunk Size",
"Min Chunk Size",
"Unique Chunks",
"Shared Chunks",
"Savings",
"Newly added (count)",
"Newly consumed (bytes)",
];
for (const profile of testProfiles) {
console.log(`Testing profile: ${profile.name}`);
for (const fn of [splitPiecesRabinKarp]) {
const funcProfile = fn !== splitPiecesRabinKarp ? "Old" : "New";
console.log(`Testing function: ${funcProfile}`);
for (const file of testFiles) {
const filePath = resolve(findPath, file);
const isPlain = shouldSplitAsPlainText(filePath);
const content = await loadFileAsBlob(filePath);
console.log(`Testing file: ${file} (size: ${content.size} bytes)`);
const result = await testSplit(fn, content, profile.settings);
const chunkSizes = result.chunks.map((c) => c.length);
const savings = result.originalSize - chunkSizes.reduce((acc, size) => acc + size, 0);
// console.log(`Result for ${file}:`, result);
results.push([
`${profile.name}`,
funcProfile,
"original",
file,
isPlain ? "plain" : "binary",
content.size.toString(),
result.chunkCount.toString(),
result.averageChunkSize.toFixed(2),
result.maxChunkSize.toString(),
result.minChunkSize.toString(),
result.uniqueChunks.toString(),
"",
savings.toString(),
"",
"",
]);
// add editions (inserting "*") to content on head, 5%, middle, 95%, tail to see if it affects the chunking
const editions = [
{ name: "head", content: modifyBlob(content, 0, "*") },
{ name: "5%", content: modifyBlob(content, Math.floor(content.size * 0.05), "*") },
{ name: "middle", content: modifyBlob(content, Math.floor(content.size * 0.5), "*") },
{ name: "95%", content: modifyBlob(content, Math.floor(content.size * 0.95), "*") },
{ name: "tail", content: modifyBlob(content, content.size, "*") },
];
const baseChunks = result.chunks;
for (const edition of editions) {
console.log(`Testing edition: ${edition.name}`);
const editionResult = await testSplit(fn, edition.content, profile.settings);
const sharedChunks = editionResult.chunks.filter((chunk) => baseChunks.includes(chunk)).length;
const newChunks = editionResult.chunks.filter((chunk) => !baseChunks.includes(chunk));
const editionResultChunkLength = editionResult.chunks.map((c) => c.length);
// console.log(`Result for edition ${edition.name} of ${file}:`, editionResult);
const editionSavings =
editionResult.originalSize - editionResultChunkLength.reduce((acc, size) => acc + size, 0);
// newly added chunks size :
const newChunksSize = newChunks.reduce((acc, chunk) => acc + chunk.length, 0);
results.push([
`${profile.name}`,
funcProfile,
`${edition.name}`,
file,
isPlain ? "plain" : "binary",
edition.content.size.toString(),
editionResult.chunkCount.toString(),
editionResult.averageChunkSize.toFixed(2),
editionResult.maxChunkSize.toString(),
editionResult.minChunkSize.toString(),
editionResult.uniqueChunks.toString(),
sharedChunks.toString(),
editionSavings.toString(),
newChunks.length.toString(),
newChunksSize.toString(),
]);
}
}
}
}
results.unshift(header);
await fs.writeFile(resolve(__dirname, "splitResults.csv"), results.map((r) => r.join(",")).join("\n"));
}
main();