Compare commits

..

202 Commits

Author SHA1 Message Date
vorotamoroz
31050c9cb8 update docs 2026-06-05 03:05:13 +09:00
vorotamoroz
1b44ab9f2b docs: update documentation 2026-06-05 01:57:03 +09:00
vorotamoroz
3a78c70539 (chore) update terms 2026-06-05 01:24:46 +09:00
vorotamoroz
0f08ad435f (chore): improve English 2026-06-05 01:08:54 +09:00
vorotamoroz
96abd930b3 Merge pull request #938 from vrtmrz/0_25_73
Release 0.25.73
2026-06-04 18:50:11 +09:00
vorotamoroz
4581db45c4 bump 2026-06-04 10:44:56 +01:00
vorotamoroz
05cab8ec66 Merge pull request #937 from vrtmrz/fix_926
fix: adjust CouchDB's database name checking to its specification
2026-06-04 18:39:38 +09:00
vorotamoroz
b005625ef3 Merge branch 'main' into fix_926 2026-06-04 18:38:26 +09:00
vorotamoroz
e5408b4dd7 Merge pull request #936 from vrtmrz/fix_path_handling_on_windows_cli
Fix: No longer path corruption on windows environment
2026-06-04 18:37:47 +09:00
vorotamoroz
95a9b1b41c Merge pull request #935 from vrtmrz/fix_first_fetch2
fixed: `Reset Syncronisation on This Device` for minio and P2P
2026-06-04 18:37:23 +09:00
vorotamoroz
2aa8bc1165 fix: adjust CouchDB's database name checking to its specification (#926). 2026-06-04 10:35:15 +01:00
vorotamoroz
f8998d5441 (fixed): No longer path corruption on windows environment (at least, pass the check) on CLI 2026-06-04 10:17:34 +01:00
vorotamoroz
26f5f54f24 fixed: Reset Syncronisation on This Device for minio and P2P is now working properly. 2026-06-04 09:40:27 +01:00
vorotamoroz
99f3aca024 Merge pull request #933 from vrtmrz/0_25_71
0 25 72
2026-06-03 20:22:29 +09:00
vorotamoroz
194397dd94 bump again 2026-06-03 12:12:21 +01:00
vorotamoroz
afa79d78dc bump 2026-06-03 11:18:25 +01:00
vorotamoroz
9a9440a768 Merge branch 'improve_first_fetch' into 0_25_71 2026-06-03 11:03:46 +01:00
vorotamoroz
e375860af8 fix importing issue and unit test issue 2026-06-03 06:58:19 +01:00
vorotamoroz
b57f34c15b Revise devs note 2026-06-03 01:13:50 +00:00
vorotamoroz
b82fd9f04b Revise redflag.md option descriptions
Updated descriptions for redflag options in troubleshooting documentation.
2026-06-02 21:14:42 +09:00
vorotamoroz
bb77426b7b update dependencies and bump 2026-06-02 12:50:46 +01:00
vorotamoroz
1bef5fbef3 Merge branch 'fix_warns' into improve_first_fetch 2026-06-02 12:36:32 +01:00
vorotamoroz
7d2ba1b0b9 ### Improved
- Database fetching (a.k.a. Reset Synchronisation on This Device) on the initialisation now supports streaming and is faster (CouchDB only)
- The database fetching process has been streamlined, and database operations are now suspended until it has been completed
- The initial synchronisation process has been simplified, making it easier to synchronise files with the remote server
- We can select the remote database to fetch from during the initialisation, when there are multiple remote databases configured (e.g. multiple CouchDBs or S3 remotes)
2026-06-02 12:34:46 +01:00
vorotamoroz
ac6b9a4dad fix bad English 2026-06-01 12:13:01 +01:00
vorotamoroz
225e2c5096 bump 2026-06-01 12:10:56 +01:00
vorotamoroz
674d68b7d9 Fixed:
-  No longer the status element breaks other plugins' interaction (#930).
2026-06-01 12:05:31 +01:00
vorotamoroz
0e6dd300ef Merge branch 'fix_warns' into improve_first_fetch 2026-06-01 11:22:01 +01:00
vorotamoroz
8171db353a bump 2026-06-01 11:19:42 +01:00
vorotamoroz
6ab1556880 prettify 2026-06-01 11:19:32 +01:00
vorotamoroz
cd2bff5fc7 Refactor types in svelte components. 2026-06-01 11:18:23 +01:00
vorotamoroz
5a280c7919 (feat): Bulk database fetching is now work in progress. This feature is expected to speed up rebuilds and setups. (WIP) 2026-06-01 10:41:48 +01:00
vorotamoroz
c6697327d5 Fixed typings
Fixed wrong typing for serviceHub, svelte dialog
2026-06-01 06:20:33 +01:00
vorotamoroz
7c203a522a Aligned to eslint rules and fixed following things:
This commit may contains behavioural changes.

- Fix for the issue with corrupted log displays
- Wrap the activeDocument
- Reduced potential type errors and strengthened certain checks
- Made error handling more robust (by rewriting the error class)
2026-06-01 05:28:03 +01:00
vorotamoroz
c80c294d93 satisfies
Address following rules
-
- @typescript-eslint/unbound-method
- obsidianmd/prefer-active-doc
- obsidianmd/prefer-window-timer

Improve typing
2026-06-01 04:20:47 +01:00
vorotamoroz
3e65ae932d Address following rules
- @typescript-eslint/no-redundant-type-constituents
2026-06-01 03:50:51 +01:00
vorotamoroz
f710f03380 Address following rules
- @typescript-eslint/no-redundant-type-constituents
- @typescript-eslint/no-unnecessary-type-assertion
2026-06-01 03:37:55 +01:00
vorotamoroz
b887269fc1 change eslint for more progressive 2026-06-01 02:37:07 +01:00
vorotamoroz
56a234e6d7 chore: wrap timer functions 2026-05-31 19:03:43 +09:00
vorotamoroz
39014b2294 use coreEnvVars for some vars 2026-05-30 23:59:36 +09:00
vorotamoroz
24e6c110a3 Remove unused imports 2026-05-30 23:53:19 +09:00
vorotamoroz
f24d110552 Change type assertion 2026-05-30 23:49:32 +09:00
vorotamoroz
7189c1c05a Update dependency
set no-deprecated to warn
2026-05-30 23:42:32 +09:00
vorotamoroz
547afe9a86 remove unused eslint comment
use _fetch instead of fetch
add ignore list
2026-05-30 23:32:46 +09:00
vorotamoroz
7b5876037d Add ignores 2026-05-30 23:22:17 +09:00
vorotamoroz
b714c00644 (chore): Improve type assertion, remove unused imports 2026-05-29 04:19:58 +01:00
vorotamoroz
e14e771bfb (chore): tidied tsconfig and eslint, and some incorrect imports 2026-05-28 04:42:03 +01:00
vorotamoroz
1130bbcee8 remove unmaintained tests 2026-05-26 11:33:14 +01:00
vorotamoroz
8841ef4619 fix css duplicated props 2026-05-26 11:27:31 +01:00
vorotamoroz
45fe0b3682 chore: lint: enable cache and concurrency 2026-05-26 11:24:41 +01:00
vorotamoroz
8d3825abc9 Merge pull request #925 from vrtmrz/v0_25_70
Releasing v0.25.70
2026-05-25 18:38:04 +09:00
vorotamoroz
c5f9841b85 chore: fix grammatical error 2026-05-25 10:14:18 +01:00
vorotamoroz
d36d176d99 bump 2026-05-25 10:10:46 +01:00
vorotamoroz
38b2cf73ed Update lib (forgot to include #942) 2026-05-25 09:55:11 +01:00
vorotamoroz
40b15a6950 Merge pull request #924 from vrtmrz/p2p_add_fix_and_diag
Improvements: More diagnostic information for P2P connections
2026-05-25 17:52:37 +09:00
vorotamoroz
e312bb7640 Merge pull request #891 from SeleiXi/feat/conflict-diff-jump
feat: add diff navigation to conflict resolver
2026-05-25 17:51:49 +09:00
Ching Wing Kwok
852c0e6c13 Merge branch 'main' into feat/conflict-diff-jump 2026-05-25 15:38:21 +08:00
vorotamoroz
7c1bcf9e9b Merge pull request #889 from SeleiXi/diff-only-button
feat: add diff-only view button to document history
2026-05-25 14:00:11 +09:00
vorotamoroz
2b79bed085 Merge branch 'main' into pr/SeleiXi/889 2026-05-25 05:43:36 +01:00
vorotamoroz
6b1e0c4aa8 Merge pull request #890 from SeleiXi/feat/history-search
feat: Add document history search
2026-05-25 12:55:29 +09:00
vorotamoroz
3c3645eba4 Improved: More diagnostic information for P2P connections is now shown, including why a connection failure occurred and the current connection status. 2026-05-25 04:31:22 +01:00
SeleiXi
009cc3c87a Merge remote-tracking branch 'origin/main' into feat/conflict-diff-jump
# Conflicts:
#	src/modules/features/InteractiveConflictResolving/ConflictResolveModal.ts
2026-05-23 02:06:52 +08:00
SeleiXi
fc5fd4be94 Merge remote-tracking branch 'origin/main' into feat/history-search
# Conflicts:
#	src/lib
#	src/modules/features/DocumentHistory/DocumentHistoryModal.ts
2026-05-23 02:05:33 +08:00
SeleiXi
8ed1acf79d Merge remote-tracking branch 'origin/main' into diff-only-button
# Conflicts:
#	src/modules/features/DocumentHistory/DocumentHistoryModal.ts
2026-05-23 02:04:48 +08:00
vorotamoroz
c518223d21 Merge pull request #923 from vrtmrz/p2p_add_fix_and_diag
0.25.69: Fix P2P, add diag feature
2026-05-23 00:47:36 +09:00
vorotamoroz
caaff618e9 fix grammar 2026-05-22 16:40:40 +01:00
vorotamoroz
148aa8505e bump 2026-05-22 16:38:44 +01:00
vorotamoroz
f9a626a858 ### Fixed
- No longer the P2P passphrase mismatch causes a server shutdown.
- Settings related to P2P synchronisation are now correctly applied on start-up and no longer reverted.

### New features
- Diagnostic P2P connection stats are now available.
  - These stats indicate the number of connection trials, successes, and, failures.
2026-05-22 16:37:05 +01:00
vorotamoroz
1b8747115c Merge pull request #922 from vrtmrz/update_trystero
Releasing v0.25.68 (dependency and P2P update)
2026-05-22 19:17:22 +09:00
vorotamoroz
e739302fb9 bump 2026-05-22 11:11:18 +01:00
vorotamoroz
8f20d53f55 Update trystero, typed well. fixed some potentially problems
Weaken terser for libraries
2026-05-22 11:01:51 +01:00
vorotamoroz
fd84b0377b Update actions 2026-05-21 10:02:46 +01:00
vorotamoroz
340d416b76 Merge pull request #920 from vrtmrz/v0_25_66
Release v0.25.67
2026-05-20 19:44:41 +09:00
vorotamoroz
3034af8d69 bump again 2026-05-20 11:37:51 +01:00
vorotamoroz
da3020bd45 fixed: fix auto-correction mismatch 2026-05-20 11:34:28 +01:00
vorotamoroz
ce232c1002 bump 2026-05-20 11:14:41 +01:00
vorotamoroz
0e13926400 fixed: update lib
I should do this with #919
2026-05-20 11:10:43 +01:00
vorotamoroz
fab7ec996a Merge pull request #919 from vrtmrz:feat_cli_remote_select
feat: add CLI commands to handle multiple remote configuration
2026-05-20 19:05:49 +09:00
vorotamoroz
88e22f99c5 Merge pull request #917 from vrtmrz:feat_tweak_auto_adjust
feat: implement auto-accept compatible tweak
2026-05-20 19:02:58 +09:00
vorotamoroz
83cbabf06f Merge branch 'main' into feat_tweak_auto_adjust 2026-05-20 11:01:51 +01:00
vorotamoroz
5e8d3b8f02 Update lib 2026-05-20 11:00:38 +01:00
vorotamoroz
1167b41340 feat: add CLI commands to handle multiple remote configuration 2026-05-20 05:10:42 +01:00
vorotamoroz
67da3964e5 Merge pull request #918 from vrtmrz/v0_25_65
V0.25.65
2026-05-19 20:27:34 +09:00
vorotamoroz
45ebc7eb6b Update the doc and issue report, and bump 2026-05-19 12:17:26 +01:00
vorotamoroz
cc5ead68bc minor layout fix 2026-05-19 11:59:54 +01:00
vorotamoroz
9b9e4f22f3 Merge pull request #916 from vrtmrz:fix_866
fixed: Now Chunk Splitter: `V3: Fine Deduplication`
2026-05-19 19:46:08 +09:00
vorotamoroz
7823f46053 Merge branch 'main' into fix_866 2026-05-19 11:45:41 +01:00
vorotamoroz
d6d8e548b3 Merge pull request #915 from vrtmrz/improve_log_feature
Improve log features
2026-05-19 19:40:28 +09:00
vorotamoroz
44b1ed7610 Merge branch 'main' into improve_log_feature 2026-05-19 11:40:14 +01:00
vorotamoroz
5786da5534 merged 2026-05-19 11:38:05 +01:00
vorotamoroz
042a80dd44 Merge pull request #914 from vrtmrz/fix_ios_resume
fix: Fix an issue about resuming from background on iOS (#888).
2026-05-19 19:34:49 +09:00
vorotamoroz
ee30f6cd6d fixed: Now Chunk Splitter: V3: Fine Deduplication is working fine again (#866). 2026-05-19 11:26:35 +01:00
vorotamoroz
977a300808 fix grammatical error 2026-05-19 04:10:39 +01:00
vorotamoroz
a392ccab6a - Improved an error verbosity on concurrent processing on start-up process.
- Now the `report` includes recent logs (of verbosity `verbose` even settings is not set to `verbose`).
- Updating logs is now debounced to avoid excessive updates during rapid log generation.
- Added a `Generate full report for opening the issue with debug info` command to the command palette, which generates a report without opening the settings dialogue.
2026-05-19 04:09:04 +01:00
vorotamoroz
dfdfa5383b grammatical fix 2026-05-19 01:55:10 +01:00
vorotamoroz
a08294ab16 fix: Fix an issue about resuming from background on iOS (#888). 2026-05-18 12:30:03 +01:00
vorotamoroz
e8c33a0d6a feat: implement auto-accept compatible tweak setting and enhance mismatch resolution logic 2026-05-18 11:21:53 +01:00
vorotamoroz
d6bf453a6d Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2026-05-18 11:07:43 +01:00
vorotamoroz
e80cdc2dae remove unused files 2026-05-18 08:47:03 +01:00
vorotamoroz
60780678fd Merge pull request #903 from Joysimple/cli-docker
bugfix: Add package-lock.json into docker build
2026-05-18 12:38:12 +09:00
vorotamoroz
273e7a2b63 Merge pull request #909 from vrtmrz/p2p_config_selectable
P2p config selectable (a.k.a. v0.25.64)
2026-05-17 13:24:40 +09:00
vorotamoroz
2572c54744 Track main 2026-05-17 13:17:09 +09:00
vorotamoroz
6a7c987985 bump 2026-05-17 13:13:21 +09:00
vorotamoroz
6ef866a77c P2P: Enhance status pane and card with active remote selection and replication features
- Added active P2P remote selector and creation option in the status pane.
- Introduced immediate replication action for accepted peers.
- Updated status control icons for clarity.
- Display stable Room ID suffix above Peer ID in the status card.
- Implemented dedicated active remote configuration for P2P features.
- Added migration support for P2P active remote selection.
- Improved unit test coverage for P2P settings.
2026-05-17 13:08:51 +09:00
vorotamoroz
eea26dee74 Update about P2P 2026-05-17 02:35:20 +09:00
vorotamoroz
ee24fe8c24 Merge pull request #904 from vrtmrz/enhance_fix_p2p
Fixed: fixed P2P bugs and and implement new UI
2026-05-17 02:32:50 +09:00
vorotamoroz
9d9364af36 Fix updates.md 2026-05-17 02:24:58 +09:00
vorotamoroz
83228e2077 Fix P2P replicator creation and enhance error handling in synchronization functions 2026-05-17 02:23:58 +09:00
vorotamoroz
a379b5bd78 bump 2026-05-17 01:40:50 +09:00
vorotamoroz
4ed1749652 Enhance P2P synchronization features and UI improvements 2026-05-17 01:36:09 +09:00
vorotamoroz
9a90256a8a Enhance P2P synchronization features and UI improvements 2026-05-16 23:50:08 +09:00
vorotamoroz
f0628a0d2c Improve UI 2026-05-16 23:09:11 +09:00
vorotamoroz
d5e2f57781 Fixed: fixed P2P bugs and and implement new UI 2026-05-15 10:18:53 +01:00
Nikolay Sokolov
02673a1631 bugfix: Add package-lock.json into docker build 2026-05-14 23:17:37 -07:00
vorotamoroz
91c9746886 Merge pull request #900 from vrtmrz/v0_25_62
Releasing v0.25.62
2026-05-14 20:15:02 +09:00
vorotamoroz
75b44b1636 bump 2026-05-14 09:40:12 +00:00
vorotamoroz
f1fe48c1ee add version-bump 2026-05-14 09:35:53 +00:00
vorotamoroz
437e7c0d9c Fixed an issue where a connection could not be established when attempting to connect to a brand-new remote database without going through the set-up wizard or configuration checking 2026-05-14 09:32:34 +00:00
vorotamoroz
5ffa7ec7ee Merge pull request #897 from vrtmrz/v0_25_61
Releaseing v0.25.61
2026-05-13 23:06:57 +09:00
vorotamoroz
a1859f5d2e bump 2026-05-13 14:44:38 +01:00
vorotamoroz
785af8cb8f Merge pull request #896 from vrtmrz/address_community_review
Address community review
2026-05-13 22:29:37 +09:00
vorotamoroz
06e1f4aa4a Update subrepo pointer 2026-05-13 14:25:49 +01:00
vorotamoroz
767f22ce9c Merge branch 'address_community_review' of https://github.com/vrtmrz/obsidian-livesync into address_community_review 2026-05-13 14:16:51 +01:00
vorotamoroz
6a9bba702c chore: ran prettier 2026-05-13 14:10:56 +01:00
vorotamoroz
de2397dc3f Adding a rough DI 2026-05-13 14:10:55 +01:00
vorotamoroz
daaad9212e Fix package-lock 2026-05-13 14:10:54 +01:00
vorotamoroz
a6891374a1 chore: Package modernise, update linter 2026-05-13 14:10:01 +01:00
vorotamoroz
b1cadf0549 prettify 2026-05-13 14:07:58 +01:00
vorotamoroz
95f40cc954 (chore): removing DOM Operation 2026-05-13 14:07:58 +01:00
vorotamoroz
8deaf123d6 Update eslint config to ignore file,
fix some type error on LiveSyncBaseCore
2026-05-13 14:06:51 +01:00
vorotamoroz
053813bffb Update for review once 2026-05-13 14:06:51 +01:00
vorotamoroz
cc7af03618 chore: Package modernise, update linter 2026-05-13 14:06:51 +01:00
vorotamoroz
a130e3700e prettify 2026-05-13 14:06:50 +01:00
vorotamoroz
0549e901b2 (chore): removing DOM Operation 2026-05-13 14:06:49 +01:00
vorotamoroz
e9afe06968 Merge pull request #895 from vrtmrz/update_lib
Update lib to fix P2P problems and merging contributions
2026-05-13 22:01:17 +09:00
vorotamoroz
c45aca4794 fixed: fixed subrepo pointer
I thought I’d rebased it, but it turns out everything had been merged.
2026-05-13 13:45:09 +01:00
vorotamoroz
e2c54aaf43 Update lib to fix P2P problems 2026-05-13 13:31:11 +01:00
vorotamoroz
37715d4c9f chore: ran prettier 2026-05-13 11:12:40 +00:00
vorotamoroz
106367fa41 Adding a rough DI 2026-05-13 11:09:04 +00:00
vorotamoroz
538130aa91 Fix package-lock 2026-05-13 11:36:01 +01:00
vorotamoroz
c9d0357fec Merge branch 'address_community_review' of https://github.com/vrtmrz/obsidian-livesync into address_community_review 2026-05-13 11:35:01 +01:00
vorotamoroz
d05c76da36 Update eslint config to ignore file,
fix some type error on LiveSyncBaseCore
2026-05-13 11:33:46 +01:00
vorotamoroz
d2eb6ecbaf Update for review once 2026-05-13 11:33:46 +01:00
vorotamoroz
25a6fde212 chore: Package modernise, update linter 2026-05-13 11:33:45 +01:00
vorotamoroz
e8f8b680ef prettify 2026-05-13 11:33:03 +01:00
vorotamoroz
6c30f2b863 (chore): removing DOM Operation 2026-05-13 11:33:03 +01:00
vorotamoroz
8dda24a689 Merge pull request #882 from vrtmrz/p2p-rpc
feat: use new p2p-rpc wrapper
2026-05-13 19:24:26 +09:00
vorotamoroz
fbbb63906a Merge branch 'main' into p2p-rpc 2026-05-13 19:22:03 +09:00
vorotamoroz
1e66a7f144 Merge pull request #894 from vrtmrz/fix_unexpected_error_on_startup
fixed: fixed unexpected error during startup
2026-05-13 19:16:18 +09:00
vorotamoroz
df79d81475 fixed: fixed unexpected error during startup 2026-05-13 10:14:47 +00:00
vorotamoroz
ad71355859 Merge pull request #893 from brian-spackman/fix-fractional-mtime-on-linux
fix: truncate sub-millisecond CLI mtimes to prevent mobile crash
2026-05-13 19:12:56 +09:00
vorotamoroz
95dc079fad Merge pull request #843 from andrewleech/daemon-sync
cli: implement continuous sync daemon mode
2026-05-13 18:53:59 +09:00
vorotamoroz
770d4af4a0 Update eslint config to ignore file,
fix some type error on LiveSyncBaseCore
2026-05-13 10:15:45 +01:00
vorotamoroz
3b311248cb Update for review once 2026-05-13 08:02:50 +01:00
Andrew Leech
67996f6d0a cli: fix stale stat.size in NodeVaultAdapter causing corrupted file errors
chokidar stats are captured at poll time and may not reflect the file's
final byte length by the time vault.read() is called. The downstream
integrity check compares stat.size to content length; a mismatch causes
other LiveSync clients to reject the file as corrupted.

Fix by updating file.stat.size from the actual content in read() and
readBinary().

Co-authored-by: Joysimple <Joysimple@users.noreply.github.com>
2026-05-13 16:56:08 +10:00
vorotamoroz
5772811a45 chore: Package modernise, update linter 2026-05-13 04:40:32 +01:00
vorotamoroz
55529cd71e prettify 2026-05-13 03:58:08 +01:00
vorotamoroz
2e9b8b7b62 (chore): removing DOM Operation 2026-05-13 03:55:11 +01:00
Andrew Leech
4ab2e41d18 cli daemon: set disableCheckingConfigMismatch for headless operation
The config mismatch dialog's defaultAction is "Dismiss" which blocks
replication. Since the daemon cannot resolve mismatches interactively,
skip the check entirely and accept the remote configuration as-is.
2026-05-13 11:21:06 +10:00
Andrew Leech
c0ad8ee15a cli: add configurable ignore rules and deployment artifacts
IgnoreRules (src/apps/cli/serviceModules/IgnoreRules.ts):
- Reads .livesync/ignore for user-defined glob patterns
- Applies gitignore matchBase semantics: patterns without / get **/ prefix,
  patterns ending with / get ** appended for directory contents
- Supports `import: .gitignore` directive to merge gitignore patterns
- Rejects negation patterns with a warning (not fully supportable)
- Integrated into both daemon and mirror commands via isTargetFile handler

Wiring:
- IgnoreRules loaded before LiveSyncBaseCore construction so beginWatch()
  receives rules when it fires during onLoad/onFirstInitialise
- Passed through initialiseServiceModulesCLI -> StorageEventManagerCLI ->
  CLIStorageEventManagerAdapter -> CLIWatchAdapter

Deployment:
- src/apps/cli/deploy/livesync-cli.service - systemd unit template
- src/apps/cli/deploy/install.sh - user/system install script

Testing:
- src/apps/cli/test/test-daemon-linux.sh - e2e tests for ignore rules
- src/apps/cli/serviceModules/IgnoreRules.unit.spec.ts - 15 unit tests
- src/apps/cli/commands/daemonCommand.unit.spec.ts - 7 unit tests
2026-05-13 11:21:06 +10:00
Andrew Leech
e6ae516493 cli: implement local→CouchDB file watching via chokidar
- Add chokidar ^4.0.0 as dependency (root package.json, runtime-package.json)
- Mark chokidar as external in vite.config.ts (not bundled, loaded at runtime)
- Implement CLIWatchAdapter.beginWatch() with chokidar:
  - ignoreInitial: true (startup files handled by mirror scan)
  - awaitWriteFinish to prevent partial-write events
  - Excludes dotfiles and .livesync/ directory at watcher level
  - Maps add/change/unlink/addDir/unlinkDir to IStorageEventWatchHandlers
  - Fatal error handler: logs clearly and releases watcher resources
- Add close() to CLIWatchAdapter, StorageEventManagerCLI for clean shutdown
- Register onUnload hook in CLIServiceModules to close watcher on shutdown
2026-05-13 11:21:06 +10:00
Andrew Leech
a4d5ef4620 cli: implement daemon startup sequence and CouchDB→local sync
- Add daemon command to help text and --interval/-i flag for polling mode
- Capture original sync settings before suspendAllSync() clobbers them
- Implement daemon startup: mirror scan → restore settings → applySettings()
  which triggers the full suspend/resume lifecycle and starts the _changes feed
- Guard processSynchroniseResult no-op to non-daemon commands so default
  handler writes incoming CouchDB changes to the local filesystem
- Polling mode: restore settings + clearInterval-safe try/catch error handling
- Warn when both liveSync and syncOnStart are false after restore (no-op config)
- Fix: only block indefinitely if daemon startup succeeded
2026-05-13 11:21:06 +10:00
Brian Spackman
3f7bb047ac fix: floor sub-millisecond CLI mtimes to prevent mobile crash
On Linux, fs.Stats.mtimeMs and ctimeMs return floats with sub-millisecond
precision derived from the kernel's nanosecond filesystem mtime. Stored
raw, this produces document timestamps like 1778511180024.462 in CouchDB
rather than integer milliseconds.

Mobile clients running LiveSync 0.25.60 have been observed to crash when
processing change-feed updates carrying non-integer millisecond timestamps
from CLI-written documents. Desktop and mobile GUI plugins write integer
milliseconds, so the crash only manifests when the headless CLI on Linux
is the source. Whether the issue was introduced in 0.25.60 or had been
latent in earlier versions hasn't been investigated; 0.25.60 is the
version where the crash was confirmed and the fix verified.

Floor the values at every stat-read site (six across three adapters and
one command) so CLI-written documents carry integer-millisecond
timestamps consistent with the rest of the mesh.
2026-05-12 18:00:25 -06:00
SeleiXi
5454e1106f feat: add diff navigation to conflict resolver 2026-05-13 00:19:56 +08:00
SeleiXi
0d9397c8b9 fix: resolve UI alignment issue for diff navigation buttons 2026-05-12 00:52:20 +08:00
SeleiXi
429a3ff1fd feat: add diff-only view button to document history 2026-05-11 23:53:07 +08:00
SeleiXi
bfff6ea7b8 feat: add document history search support 2026-05-11 22:45:42 +08:00
vorotamoroz
b6b153c0de Merge pull request #887 from vrtmrz/add_ignore_to_eslint
chore: Change eslint config to ignore _tools
2026-05-11 21:01:47 +09:00
vorotamoroz
eca6a6e0ba chore: Change eslint config to ignore _tools 2026-05-11 13:00:32 +01:00
vorotamoroz
ca43d96c46 Merge pull request #886 from vrtmrz/fix_prettier
Fix prettier config
2026-05-11 20:34:22 +09:00
vorotamoroz
112e3c8b1d Fix prettier config 2026-05-11 12:33:32 +01:00
vorotamoroz
d1eb105801 Merge pull request #872 from OriBoharon/make-cli-onboarding-easier
added documentaion and a hook build script to make onbaording easier when trying to build the cli app
2026-05-11 18:43:00 +09:00
vorotamoroz
d5b93e89cd Change the default issue report label from 'bug' to 'uncategorised' 2026-05-11 17:55:31 +09:00
vorotamoroz
e96fe7cde1 Merge pull request #885 from vrtmrz/tidy_file
(chore) remove obsoleted file
2026-05-11 17:52:22 +09:00
vorotamoroz
68e0610f1d (chore) remove obsoleted file 2026-05-11 09:49:32 +01:00
vorotamoroz
a6be20695a feat: use new p2p-rpc wrapper 2026-05-11 03:49:35 +01:00
vorotamoroz
772b6ecf26 Merge pull request #871 from SeleiXi/feat/diff-navigation-buttons
feat: Add diff navigation buttons for Document History
2026-05-09 22:51:04 +09:00
SeleiXi
81dc7f604b feat: auto navigation to diff 2026-05-09 14:07:08 +08:00
vorotamoroz
a9c87fa52e - Add default test environment
- Fixed to use environment by APIs
- Make test parallel
2026-05-08 03:04:14 +00:00
vorotamoroz
e81f023943 Add default test env 2026-05-08 03:01:22 +00:00
vorotamoroz
2afe12ad2d fix pattern 2026-05-07 11:28:01 +01:00
vorotamoroz
4a9d6c1349 Add ci 2026-05-07 11:23:51 +01:00
vorotamoroz
279fc8876e feat(tests): enhance push/pull test with Docker integration and improved environment variable handling
style(test): format comment
2026-05-07 11:22:56 +01:00
vorotamoroz
cc3d30dbcf feat(tests): add Deno-based tests for checking CLI functionality in the same-codebase between platforms. 2026-05-07 11:06:12 +01:00
vorotamoroz
39e82cc8a1 Fixed: Fix timing issue during test 2026-05-06 21:56:13 +09:00
bori
7a4b76a550 added documentaion and a hook build script to make onbaording easier 2026-05-02 18:51:07 +03:00
SeleiXi
f9294446ba feat: add diff block navigation to Document History modal
Add prev/next buttons to jump between diff blocks in the
Document History view. Includes position indicator and
auto-scroll with visual focus highlighting.
2026-05-02 22:18:43 +08:00
vorotamoroz
fa7ef62302 Fix: adjusting help
Co-authored-by: Copilot <copilot@github.com>
2026-04-29 18:42:54 +09:00
vorotamoroz
81d8224330 bump
Co-authored-by: Copilot <copilot@github.com>
2026-04-29 18:39:48 +09:00
vorotamoroz
cc466a4b3c ### Fixed
- Now larger settings can be exported and imported via QR code without issues. (#595)

- Fixed some errors during serialisation and deserialisation of the settings, which caused issues in some cases when importing/exporting settings via QR code.

Co-authored-by: Copilot <copilot@github.com>
2026-04-29 18:37:44 +09:00
vorotamoroz
ceebca7de9 Merge pull request #862 from fabiomanz/main
chore: remove obsolete `version` attribute from docker-compose.yml
2026-04-29 17:30:35 +09:00
Fabio
c2f696d0a4 chore: attribute version is obsolete 2026-04-29 07:07:45 +00:00
vorotamoroz
1aa7c45794 Fix the readme
Co-authored-by: Copilot <copilot@github.com>
2026-04-29 12:55:34 +09:00
vorotamoroz
faefa80cbd Fix again 2026-04-29 12:40:40 +09:00
vorotamoroz
3737eacffd Fix readme
Co-authored-by: Copilot <copilot@github.com>
2026-04-29 12:39:42 +09:00
vorotamoroz
4c0af0b608 Fixed(cli):
- `ls` and `mirror` commands now provide informative feedback when no documents are found or filters skip all files, resolving the issue where they would exit silently (#860).
- The command-line argument `vault` has been renamed to a more appropriate name, `databaseDir`.
- The `mirror` command now accepts a `vault` directory, which specifies the location where the actual files are stored. For compatibility reasons, the previous behaviour is still supported.

Co-authored-by: Copilot <copilot@github.com>
2026-04-29 12:22:00 +09:00
vorotamoroz
bb69eb13e7 bump 2026-04-27 11:15:07 +09:00
vorotamoroz
7c9db6376f Fixed:
- No longer Setup-wizard drops username and password silently. (#865)
- Setup URI is now correctly imported (#859).
- now French translation is added.
2026-04-27 11:14:06 +09:00
vorotamoroz
4c04e4e676 Merge pull request #863 from koteitan/fix/859-strip-trailing-slash-from-uri
fix: strip trailing slash from couchDB_URI to avoid double-slash 401
2026-04-27 11:08:11 +09:00
koteitan
14ec35b257 fix: strip trailing slash from couchDB_URI to avoid double-slash 401
When couchDB_URI ends with a trailing slash (e.g. https://host/), the
database name concatenation produces a double-slash path
(https://host//obsidiannotes), which causes CouchDB to reject requests
with 401 "Name or password is incorrect".

Strip trailing slashes from couchDB_URI / baseUri at the path
concatenation sites in:
- src/common/utils.ts (_requestToCouchDBFetch, _requestToCouchDB)
- src/features/LocalDatabaseMainte/CmdLocalDatabaseMainte.ts

The companion fix for the replication path is in the livesync-commonlib
submodule.

Ref: #859
2026-04-27 00:12:57 +09:00
vorotamoroz
b609e4973c Merge remote-tracking branch 'refs/remotes/origin/main' 2026-04-25 20:37:08 +09:00
vorotamoroz
16804ed34c Merge pull request #842 from kdavh/patch-1
Update README.md, fix webpeer link
2026-04-25 19:07:56 +09:00
kdavh
12f04f6cf7 Update README.md, fix webpeer link 2026-03-28 12:47:28 -04:00
175 changed files with 13965 additions and 4157 deletions

View File

@@ -2,7 +2,7 @@
name: Issue report
about: Create a report to help us improve
title: ''
labels: 'bug'
labels: 'uncategorised'
assignees: ''
---
@@ -53,11 +53,12 @@ The hatch report (below) includes version information. If you cannot provide the
- Self-hosted LiveSync version: <!-- e.g. 0.23.0 — find it in Obsidian Settings → Community Plugins -->
### Report from LiveSync
Open the `Hatch` pane in LiveSync settings and press `Make report`. Paste here or upload to [Gist](https://gist.github.com/) and share the link.
### Report and Logs from LiveSync
Perform a `Generate full report for opening the issue with debug info` command and provide the generated report. This contains detailed information and recent 1000 log lines, which is very helpful for debugging. **PLEASE AMEND THE REPORT TO REMOVE ANY SENSITIVE INFORMATION BEFORE PASTING.**
If too large to paste here, upload to [Gist](https://gist.github.com/) and share the link.
<details>
<summary>Report from hatch (primary)</summary>
<summary>Report and Logs (primary)</summary>
```
<!-- paste here or link to Gist -->
@@ -65,29 +66,7 @@ Open the `Hatch` pane in LiveSync settings and press `Make report`. Paste here o
</details>
<details>
<summary>Report from hatch (if applicable)</summary>
```
<!-- paste here or link to Gist -->
```
</details>
### Plug-in log
Enable `Verbose Log` in General Settings first, then reproduce the issue and copy the log (tap the document box icon in the ribbon).
Paste here or upload to [Gist](https://gist.github.com/) and share the link.
<details>
<summary>Plug-in log (primary)</summary>
```
<!-- paste here or link to Gist -->
```
</details>
<details>
<summary>Plug-in log (if applicable)</summary>
<summary>Report and Logs (if applicable)</summary>
```
<!-- paste here or link to Gist -->

114
.github/workflows/cli-deno-tests.yml vendored Normal file
View File

@@ -0,0 +1,114 @@
name: cli-deno-tests
on:
workflow_dispatch:
inputs:
test_task:
description: 'Deno test task to run'
type: choice
options:
- test
- test:local
- test:e2e-matrix
- test:p2p-sync
default: test
permissions:
contents: read
jobs:
prepare:
runs-on: ubuntu-latest
outputs:
task_matrix: ${{ steps.select.outputs.task_matrix }}
steps:
- name: Select task matrix
id: select
shell: bash
run: |
set -euo pipefail
SELECTED_TASK="${{ github.event_name == 'workflow_dispatch' && inputs.test_task || 'test' }}"
echo "[INFO] Selected task set: $SELECTED_TASK"
case "$SELECTED_TASK" in
test)
TASK_MATRIX='["test:setup-put-cat","test:mirror","test:push-pull","test:sync-two-local","test:sync-locked-remote","test:p2p-host","test:p2p-peers","test:p2p-sync","test:p2p-three-nodes","test:p2p-upload-download","test:e2e-couchdb","test:e2e-matrix"]'
;;
test:local)
TASK_MATRIX='["test:setup-put-cat","test:mirror"]'
;;
test:e2e-matrix)
TASK_MATRIX='["test:e2e-matrix"]'
;;
test:p2p-sync)
TASK_MATRIX='["test:p2p-sync"]'
;;
*)
echo "[ERROR] Unknown task set: $SELECTED_TASK" >&2
exit 1
;;
esac
echo "task_matrix=$TASK_MATRIX" >> "$GITHUB_OUTPUT"
test:
needs: prepare
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
task: ${{ fromJson(needs.prepare.outputs.task_matrix) }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '24.x'
cache: 'npm'
- name: Setup Deno
uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- name: Install dependencies
run: npm ci
- name: Build CLI
working-directory: src/apps/cli
run: npm run build
- name: Create .test.env
working-directory: src/apps/cli
run: |
cat <<EOF > .test.env
hostname=http://127.0.0.1:5989/
dbname=livesync-test-db-ci
username=admin
password=testpassword
minioEndpoint=http://127.0.0.1:9000
accessKey=minioadmin
secretKey=minioadmin
bucketName=livesync-test-bucket-ci
EOF
- name: Run Deno tests
working-directory: src/apps/cli/testdeno
env:
LIVESYNC_DOCKER_MODE: native
LIVESYNC_CLI_RETRY: 3
run: |
TASK="${{ matrix.task }}"
echo "[INFO] Running Deno task: $TASK"
deno task "$TASK"
- name: Stop leftover containers
if: always()
run: |
docker stop couchdb-test minio-test relay-test >/dev/null 2>&1 || true
docker rm couchdb-test minio-test relay-test >/dev/null 2>&1 || true

View File

@@ -9,6 +9,10 @@ on:
jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: write
id-token: write
attestations: write
steps:
- uses: actions/checkout@v4
with:
@@ -29,68 +33,20 @@ jobs:
run: |
npm ci
npm run build --if-present
# Attest
- name: Attest Plugin Artifacts
uses: actions/attest-build-provenance@v4
with:
subject-path: |
main.js
manifest.json
styles.css
# Package the required files into a zip
- name: Package
run: |
mkdir ${{ github.event.repository.name }}
cp main.js manifest.json styles.css README.md ${{ github.event.repository.name }}
zip -r ${{ github.event.repository.name }}.zip ${{ github.event.repository.name }}
# Create the release on github
# - name: Create Release
# id: create_release
# uses: actions/create-release@v1
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# VERSION: ${{ steps.version.outputs.tag }}
# with:
# tag_name: ${{ steps.version.outputs.tag }}
# release_name: ${{ steps.version.outputs.tag }}
# draft: true
# prerelease: false
# # Upload the packaged release file
# - name: Upload zip file
# id: upload-zip
# uses: actions/upload-release-asset@v1
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.create_release.outputs.upload_url }}
# asset_path: ./${{ github.event.repository.name }}.zip
# asset_name: ${{ github.event.repository.name }}-${{ steps.version.outputs.tag }}.zip
# asset_content_type: application/zip
# # Upload the main.js
# - name: Upload main.js
# id: upload-main
# uses: actions/upload-release-asset@v1
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.create_release.outputs.upload_url }}
# asset_path: ./main.js
# asset_name: main.js
# asset_content_type: text/javascript
# # Upload the manifest.json
# - name: Upload manifest.json
# id: upload-manifest
# uses: actions/upload-release-asset@v1
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.create_release.outputs.upload_url }}
# asset_path: ./manifest.json
# asset_name: manifest.json
# asset_content_type: application/json
# # Upload the style.css
# - name: Upload styles.css
# id: upload-css
# uses: actions/upload-release-asset@v1
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.create_release.outputs.upload_url }}
# asset_path: ./styles.css
# asset_name: styles.css
# asset_content_type: text/css
- name: Create Release and Upload Assets
uses: softprops/action-gh-release@v2
with:

View File

@@ -10,7 +10,18 @@ on:
paths:
- 'src/**'
- 'test/**'
- 'lib/**'
- 'package.json'
- 'package-lock.json'
- 'tsconfig.json'
- 'vite.config.ts'
- 'vitest.config*.ts'
- 'esbuild.config.mjs'
- 'eslint.config.mjs'
- '.github/workflows/unit-ci.yml'
pull_request:
paths:
- 'src/**'
- 'test/**'
- 'package.json'
- 'package-lock.json'
- 'tsconfig.json'

5
.gitignore vendored
View File

@@ -28,4 +28,7 @@ data.json
cov_profile/**
coverage
src/apps/cli/dist/*
src/apps/cli/dist/*
_testdata/**
utils/bench/splitResults.csv
.eslintcache

View File

@@ -13,7 +13,7 @@ const prettierConfig = {
tabWidth: 4,
printWidth: 120,
semi: true,
endOfLine: "cr",
endOfLine: "lf",
...localPrettierConfig,
};

65
AGENTS.md Normal file
View File

@@ -0,0 +1,65 @@
# AI Coding Assistant Instructions (AGENTS.md)
When working on this repository (writing code, comments, documentation, or commits), you MUST follow these guidelines to maintain consistency.
## Required Reference Files
Before making changes to documentation, user-facing text, or settings:
1. Read [docs/terms.md](file:///Users/vorotamoroz/dev/js/obsidian-livesync/docs/terms.md) for terminology, vocabulary conventions, and technical definitions.
2. Read [docs/settings.md](file:///Users/vorotamoroz/dev/js/obsidian-livesync/docs/settings.md) (and [docs/settings_ja.md](file:///Users/vorotamoroz/dev/js/obsidian-livesync/docs/settings_ja.md)) for UI settings and setting key mappings.
3. Read [docs/troubleshooting.md](file:///Users/vorotamoroz/dev/js/obsidian-livesync/docs/troubleshooting.md) for troubleshooting guidelines and common recovery steps (such as flag files and SCRAM state).
4. Read [devs.md](file:///Users/vorotamoroz/dev/js/obsidian-livesync/devs.md) for development workflows, module architecture, and testing infrastructure.
---
## Documentation and User-Facing Text Rules
Always adhere to the following stylistic and spelling rules:
1. **British English Spelling**:
- Write all documentation and user-facing messages in British English. If in doubt, the BBC News Styleguide may be useful as a reference.
- **Traditional Spelling (Trad-spelling)**: Use `-ise` and `-isation` suffixes instead of `-ize` and `-ization` (for example: 'initialisation', 'synchronisation', and 'organisation').
- **Oxford Comma**: Use the serial (Oxford) comma to separate items in lists of three or more (for example: 'settings, snippets, and themes' instead of 'settings, snippets and themes').
- **Logical Punctuation**: Place punctuation marks (such as commas and full stops) outside quotation marks unless they are part of the quoted text itself (for example: write 'dialogue', not 'dialogue,').
2. **No Contractions**:
- Do not use contractions in general text or documentation (for example: write "do not" instead of "don't", "cannot" instead of "can't", and "is not" instead of "isn't").
3. **Quotation Style**:
- Prefer single quotation marks (`'`) over double quotation marks (`"`) in general documentation text, unless the context requires double quotes (for example, inside JSON code blocks).
4. **Specific Terminology and Spelling**:
- Use **'dialogue'** in documentation, user-facing messages, and general text. Use **'dialog'** only inside source code (e.g. class names, methods).
- Use the hyphenated form **'plug-in'** in user-facing text. Use **'plugin'** only in codebase files, configuration settings, or technical contexts.
---
## Technical & Architecture Rules
1. **Database Structure**:
- Remember that Self-hosted LiveSync splits files into **Metadata** (file properties, size, paths) and **Chunks** (actual content). Do not store raw content in the metadata document directly.
2. **Setup and Recovery**:
- **Fast Setup (Simple Fetch)** is the preferred flow for initial replication on secondary devices. It utilises stream-based replication for high speed and delays local file reflection to suppress temporary synchronisation warnings.
- **Flag files** (such as `redflag.md`, `redflag2.md`, and `redflag3.md`) at the root of the vault control the boot-up sequence and trigger automated fetch/rebuild tasks.
3. **Subrepositories**:
- The directory [src/lib](file:///Users/vorotamoroz/dev/js/obsidian-livesync/src/lib) is a subrepository (Git submodule) pointing to the shared library `livesync-commonlib`. Do not make modifications inside this directory without careful consideration, as changes affect the shared library.
4. **Application Directories**:
- The directory [src/apps](file:///Users/vorotamoroz/dev/js/obsidian-livesync/src/apps) contains independent application modules:
- `cli`: A Command Line Interface application. Tests specifically for the CLI (both unit and End-to-End tests) are located and executed within [src/apps/cli](file:///Users/vorotamoroz/dev/js/obsidian-livesync/src/apps/cli) using its local `package.json` scripts.
- `webapp`: A Web-based application.
- `webpeer`: A Web-based peer utility.
---
## Development & Verification Commands
Before submitting code, you should run verification scripts locally to ensure correct syntax and function.
1. **Lint and Type Checking**:
- Run `npm run check` to perform code verification. This runs type-checking (`tsc-check`), ESLint (`lint`), and Svelte checks (`svelte-check`).
2. **Unit Tests**:
- Run `npm run test:unit` to execute fast local unit tests.
- Run `npm run test` or `npm run test:full` for full testing suites (including dockerised services).
3. **Build**:
- Run `npm run build` to compile the production bundle (`main.js`).
- Run `npm run dev` for the development watch/build task.

View File

@@ -4,7 +4,7 @@
Self-hosted LiveSync is a community-developed synchronisation plug-in available on all Obsidian-compatible platforms. It leverages robust server solutions such as CouchDB or object storage systems (e.g., MinIO, S3, R2, etc.) to ensure reliable data synchronisation.
Additionally, it supports peer-to-peer synchronisation using WebRTC now (experimental), enabling you to synchronise your notes directly between devices without relying on a server.
Additionally, it supports peer-to-peer synchronisation using WebRTC, enabling you to synchronise your notes directly between devices without relying on a server. Documentation is available for [Peer-to-Peer Synchronisation](./docs/p2p_sync_updates_2026.md).
![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
@@ -24,18 +24,18 @@ Additionally, it supports peer-to-peer synchronisation using WebRTC now (experim
- WebRTC is a peer-to-peer synchronisation method, so **at least one device must be online to synchronise**.
- Instead of keeping your device online as a stable peer, you can use two pseudo-peers:
- [livesync-serverpeer](https://github.com/vrtmrz/livesync-serverpeer): A pseudo-client running on the server for receiving and sending data between devices.
- [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer): A pseudo-client for receiving and sending data between devices.
- A pre-built instance is available at [fancy-syncing.vrtmrz.net/webpeer](https://fancy-syncing.vrtmrz.net/webpeer/) (hosted on the vrtmrz blog site). This is also peer-to-peer. Feel free to use it.
- [webpeer](https://github.com/vrtmrz/obsidian-livesync/tree/main/src/apps/webpeer): A pseudo-client for receiving and sending data between devices.
- A pre-built instance is available at [fancy-syncing.vrtmrz.net/webpeer](https://fancy-syncing.vrtmrz.net/webpeer/) (hosted on the vrtmrz's blog site). This is also peer-to-peer. Feel free to use it.
- For more information, refer to the [English explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync-en.html) or the [Japanese explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync).
This plug-in may be particularly useful for researchers, engineers, and developers who need to keep their notes fully self-hosted for security reasons. It is also suitable for anyone seeking the peace of mind that comes with knowing their notes remain entirely private.
>[!IMPORTANT]
> - Before installing or upgrading this plug-in, please back up your vault.
> - Do not enable this plug-in alongside another synchronisation solution at the same time (including iCloud and Obsidian Sync).
> - Do not enable this plug-in alongside another synchronisation solution (including iCloud and Obsidian Sync).
> - For backups, we also provide a plug-in called [Differential ZIP Backup](https://github.com/vrtmrz/diffzip).
## How to use
## How to Use
### 3-minute setup - CouchDB on fly.io
@@ -43,54 +43,55 @@ This plug-in may be particularly useful for researchers, engineers, and develope
[![LiveSync Setup onto Fly.io SpeedRun 2024 using Google Colab](https://img.youtube.com/vi/7sa_I1832Xc/0.jpg)](https://www.youtube.com/watch?v=7sa_I1832Xc)
1. [Setup CouchDB on fly.io](docs/setup_flyio.md)
1. [Set up CouchDB on fly.io](docs/setup_flyio.md)
2. Configure plug-in in [Quick Setup](docs/quick_setup.md)
### Manually Setup
### Manual Setup
1. Setup the server
1. [Setup CouchDB on fly.io](docs/setup_flyio.md)
2. [Setup your CouchDB](docs/setup_own_server.md)
1. Set up the server
1. [Set up CouchDB on fly.io](docs/setup_flyio.md)
2. [Set up your CouchDB](docs/setup_own_server.md)
2. Configure plug-in in [Quick Setup](docs/quick_setup.md)
> [!TIP]
> Fly.io is no longer free. Fortunately, despite some issues, we can still use IBM Cloudant. Refer to [Setup IBM Cloudant](docs/setup_cloudant.md).
> And also, we can use peer-to-peer synchronisation without a server. Or very cheap Object Storage -- Cloudflare R2 can be used for free.
> HOWEVER, most importantly, we can use the server that we trust. Therefore, please set up your own server.
> CouchDB can be run on a Raspberry Pi. (But please be careful about the security of your server).
> Fly.io is no longer free. Fortunately, we can still use IBM Cloudant despite some limitations. Refer to [Set up IBM Cloudant](docs/setup_cloudant.md).
> We can also use peer-to-peer synchronisation without a server. Alternatively, cheap object storage like Cloudflare R2 can be used for free.
> However, most importantly, we can use a server that we trust. Therefore, please set up your own server.
> CouchDB can also be run on a Raspberry Pi (please be mindful of your server's security).
## Information in StatusBar
## Information in the Status Bar
Synchronization status is shown in the status bar with the following icons.
Synchronisation status is shown in the status bar with the following icons.
- Activity Indicator
- 📲 Network request
- Status
- ⏹️ Stopped
- 💤 LiveSync enabled. Waiting for changes
- ⚡️ Synchronization in progress
- ⚡️ Synchronisation in progress
- ⚠ An error occurred
- Statistical indicator
- Statistical Indicators
- ↑ Uploaded chunks and metadata
- ↓ Downloaded chunks and metadata
- Progress indicator
- Progress Indicators
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- 📬 Batched read storage processes
- ⚙️ Working or pending storage processes of hidden files
- ⚙️ Working or pending storage processes for hidden files
- 🧩 Waiting chunks
- 🔌 Working Customisation items (Configuration, snippets, and plug-ins)
- 🔌 Working customisation items (configuration, snippets, and plug-ins)
To prevent file and database corruption, please wait to stop Obsidian until all progress indicators have disappeared as possible (The plugin will also try to resume, though). Especially in case of if you have deleted or renamed files.
To prevent file and database corruption, please avoid closing Obsidian until all progress indicators have disappeared as much as possible (although the plug-in will attempt to resume if interrupted). This is especially important if you have deleted or renamed files.
## Tips and Troubleshooting
If you are having problems getting the plugin working see: [Tips and Troubleshooting](docs/troubleshooting.md).
- If you want a faster and simpler initial replication when setting up subsequent devices, see the [Fast Setup Guide](docs/tips/fast-setup.md).
- If you are having problems getting the plug-in working, see [Tips and Troubleshooting](docs/troubleshooting.md).
## Acknowledgements
The project has been in continual progress and harmony thanks to:
The project has been in continual progress and harmony thanks to the following:
- Many [Contributors](https://github.com/vrtmrz/obsidian-livesync/graphs/contributors).
- Many [GitHub Sponsors](https://github.com/sponsors/vrtmrz#sponsors).
- JetBrains Community Programs / Support for Open-Source Projects. <img src="https://resources.jetbrains.com/storage/products/company/brand/logos/jetbrains.png" alt="JetBrains logo" height="24">
@@ -98,7 +99,7 @@ The project has been in continual progress and harmony thanks to:
May those who have contributed be honoured and remembered for their kindness and generosity.
## Development Guide
Please refer to [Development Guide](devs.md) for development setup, testing infrastructure, code conventions, and more.
Please refer to the [Development Guide](devs.md) for development setup, testing infrastructure, code conventions, and more.
## License

View File

@@ -78,7 +78,8 @@ NDAや類似の契約や義務、倫理を守る必要のある、研究者、
## Tips and Troubleshooting
何かこまったら、[Tips and Troubleshooting](docs/troubleshooting.md)をご参照ください。
- 2台目以降のセットアップ時に、初期同期をより迅速かつ簡単に行うには、[ファストセットアップガイド](docs/tips/fast-setup_ja.md)をご参照ください。
- 何かこまったら、[Tips and Troubleshooting](docs/troubleshooting.md)をご参照ください。
## License

92
aggregator.html Normal file
View File

@@ -0,0 +1,92 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Self-hosted LiveSync Setup QR Aggregator</title>
<style>
body { font-family: sans-serif; display: flex; flex-direction: column; align-items: center; justify-content: center; height: 100vh; margin: 0; background-color: #f4f4f9; color: #333; }
.container { background: white; padding: 2rem; border-radius: 8px; box-shadow: 0 4px 6px rgba(0,0,0,0.1); text-align: center; max-width: 90%; }
.progress { margin: 20px 0; font-size: 1.2rem; font-weight: bold; }
.status { margin-bottom: 20px; color: #666; }
.btn { display: inline-block; padding: 12px 24px; background-color: #7c4dff; color: white; text-decoration: none; border-radius: 4px; font-weight: bold; transition: background-color 0.2s; border: none; cursor: pointer; }
.btn:hover { background-color: #651fff; }
.btn:disabled { background-color: #ccc; cursor: not-allowed; }
.grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(40px, 1fr)); gap: 8px; margin: 20px 0; }
.tile { width: 40px; height: 40px; border: 2px solid #ddd; border-radius: 4px; display: flex; align-items: center; justify-content: center; font-size: 0.8rem; }
.tile.filled { background-color: #7c4dff; color: white; border-color: #7c4dff; }
</style>
</head>
<body>
<div class="container">
<h1>LiveSync Setup</h1>
<div id="app">
<p>Checking hash data...</p>
</div>
</div>
<script>
function updateUI() {
const hash = window.location.hash.substring(1);
const params = new URLSearchParams(hash);
const id = params.get('id');
const total = parseInt(params.get('n') || '0');
const index = parseInt(params.get('i') || '-1');
const data = params.get('d');
const app = document.getElementById('app');
if (!id || total <= 0 || index === -1 || !data) {
app.innerHTML = '<p class="status">Invalid setup URL. Please scan the QR code correctly.</p>';
return;
}
// Get session data
const storageKey = 'ls_agg_' + id;
let session = JSON.parse(localStorage.getItem(storageKey) || '{}');
// Save current data
session[index] = data;
localStorage.setItem(storageKey, JSON.stringify(session));
const receivedIndexes = Object.keys(session).map(Number);
const count = receivedIndexes.length;
let html = `
<div class="status">Session ID: ${id}</div>
<div class="progress">${count} / ${total} Loaded</div>
<div class="grid">
`;
for (let i = 0; i < total; i++) {
const isFilled = session[i] !== undefined;
html += `<div class="tile ${isFilled ? 'filled' : ''}">${i + 1}</div>`;
}
html += `</div>`;
if (count === total) {
const sortedData = Array.from({length: total}, (_, i) => session[i]).join('');
// Use the correct protocol for settings
const obsidianUri = `obsidian://setuplivesync?settingsQR=${sortedData}`;
html += `
<p>All parts have been collected!</p>
<a href="${obsidianUri}" class="btn">Open Obsidian to complete setup</a>
<p style="margin-top:20px; font-size:0.8rem; color: #999;">Note: If the button does not respond, please ensure you are opening this in a browser that can trigger Obsidian.</p>
`;
} else {
html += `
<p class="status">Please scan the next QR code.</p>
<button class="btn" disabled>Waiting...</button>
`;
}
app.innerHTML = html;
}
window.addEventListener('hashchange', updateUI);
updateUI();
</script>
</body>
</html>

121
devs.md
View File

@@ -3,6 +3,76 @@
Self-hosted LiveSync is an Obsidian plugin for synchronising vaults across devices using CouchDB, MinIO/S3, or peer-to-peer WebRTC. The codebase uses a modular architecture with TypeScript, Svelte, and PouchDB.
## Build & Development Workflow
### Environment Setup
#### First-time Setup
This repository uses submodules by convention. Therefore, you must use the `--recursive` flag when cloning it.
```bash
git clone --recursive https://github.com/vrtmrz/obsidian-livesync
npm ci
npm run build
```
Note: if you already cloned without submodules, run: `git submodule update --init --recursive`
#### Branch switching
When switching branches, please make sure to update submodules as well, since they may be updated in the new branch.
```bash
git checkout --recurse-submodules 0.25.70-patch1 # tag or branch name
npm ci
npm run build
```
### Commands
```bash
npm run test:unit # Run unit tests with vitest (or `npm run test:unit:coverage` for coverage)
npm run check # TypeScript and svelte type checking
npm run dev # Development build with auto-rebuild (uses .env for test vault paths)
npm run build # Production build
npm run buildDev # Development build (one-time)
npm run bakei18n # Pre-build step: compile i18n resources (YAML → JSON → TS)
npm run test:unit # Run unit tests only (no Docker services required)
npm test # Run Harness based vitest tests (requires Docker services), not recommended, unstable.
```
### Tips
We can use CLI's E2E test command instead of `npm test`.
### Auto-copy to test vaults
To facilitate development and testing, the build process can automatically copy the built plugin to specified test vault
- Create `.env` file with `PATHS_TEST_INSTALL` pointing to test vault plug-in directories (`:` separated on Unix, `;` on Windows)
- Development builds auto-copy to these paths on build whilst `npm run dev` is running (watch mode)
### Testing Infrastructure
- ~~**Deno Tests**: Unit tests for platform-independent code (e.g., `HashManager.test.ts`)~~
- This is now obsolete, migrated to vitest.
- **Vitest** (`vitest.config.ts`): E2E test by Browser-based-harness using Playwright, unit tests.
- Unit tests should be `*.unit.spec.ts` and placed alongside the implementation file (e.g., `ChunkFetcher.unit.spec.ts`).
- **Docker Services**: Tests require CouchDB, MinIO (S3), and P2P services:
```bash
npm run test:docker-all:start # Start all test services
npm run test:full # Run tests with coverage
npm run test:docker-all:stop # Stop services
```
If some services are not needed, start only required ones (e.g., `test:docker-couchdb:start`)
Note that if services are already running, starting script will fail. Please stop them first.
- **Test Structure**:
- `test/suite/` - Integration tests for sync operations
- `test/unit/` - Unit tests (via vitest, as harness is browser-based)
- `test/harness/` - Mock implementations (e.g., `obsidian-mock.ts`)
## Architecture
### Module System
@@ -17,7 +87,7 @@ The plugin uses a dynamic module system to reduce coupling and improve maintaina
- `coreObsidian/` - Obsidian-specific core (e.g., `ModuleFileAccessObsidian`)
- `essential/` - Required modules (e.g., `ModuleMigration`, `ModuleKeyValueDB`)
- `features/` - Optional features (e.g., `ModuleLog`, `ModuleObsidianSettings`)
- `extras/` - Development/testing tools (e.g., `ModuleDev`, `ModuleIntegratedTest`)
- `extras/` - Development/testing tools (e.g., `ModuleDev`, ~~`ModuleIntegratedTest`~~)
- **Services**: Core services (e.g., `database`, `replicator`, `storageAccess`) are registered in `ServiceHub` and accessed by modules. They provide an extension point for add new behaviour without modifying existing code.
- For example, checks before the replication can be added to the `replication.onBeforeReplicate` handler, and the handlers can be return `false` to prevent replication-starting. `vault.isTargetFile` also can be used to prevent processing specific files.
- **ServiceModule**: A new type of module that directly depends on services.
@@ -47,45 +117,6 @@ Hence, the new feature should be implemented as follows:
- **Development code**: Use `.dev.ts` suffix (replaced with `.prod.ts` in production)
- **Path aliases**: `@/*` maps to `src/*`, `@lib/*` maps to `src/lib/src/*`
## Build & Development Workflow
### Commands
```bash
npm run test:unit # Run unit tests with vitest (or `npm run test:unit:coverage` for coverage)
npm run check # TypeScript and svelte type checking
npm run dev # Development build with auto-rebuild (uses .env for test vault paths)
npm run build # Production build
npm run buildDev # Development build (one-time)
npm run bakei18n # Pre-build step: compile i18n resources (YAML → JSON → TS)
npm test # Run vitest tests (requires Docker services)
```
### Environment Setup
- Create `.env` file with `PATHS_TEST_INSTALL` pointing to test vault plug-in directories (`:` separated on Unix, `;` on Windows)
- Development builds auto-copy to these paths on build
### Testing Infrastructure
- ~~**Deno Tests**: Unit tests for platform-independent code (e.g., `HashManager.test.ts`)~~
- This is now obsolete, migrated to vitest.
- **Vitest** (`vitest.config.ts`): E2E test by Browser-based-harness using Playwright, unit tests.
- Unit tests should be `*.unit.spec.ts` and placed alongside the implementation file (e.g., `ChunkFetcher.unit.spec.ts`).
- **Docker Services**: Tests require CouchDB, MinIO (S3), and P2P services:
```bash
npm run test:docker-all:start # Start all test services
npm run test:full # Run tests with coverage
npm run test:docker-all:stop # Stop services
```
If some services are not needed, start only required ones (e.g., `test:docker-couchdb:start`)
Note that if services are already running, starting script will fail. Please stop them first.
- **Test Structure**:
- `test/suite/` - Integration tests for sync operations
- `test/unit/` - Unit tests (via vitest, as harness is browser-based)
- `test/harness/` - Mock implementations (e.g., `obsidian-mock.ts`)
## Code Conventions
### Internationalisation (i18n)
@@ -153,17 +184,17 @@ export class ModuleExample extends AbstractObsidianModule {
## Beta Policy
- Beta versions are denoted by appending `+patchedN` to the base version number.
- Beta versions are denoted by appending `-patchedN` to the base version number.
- `The base version` mostly corresponds to the stable release version.
- e.g., v0.25.41+patched1 is equivalent to v0.25.42-beta1.
- e.g., v0.25.41-patched1 is equivalent to v0.25.42-beta1.
- This notation is due to SemVer incompatibility of Obsidian's plugin system.
- Hence, this release is `0.25.41+patched1`.
- Hence, this release is `0.25.41-patched1`.
- Each beta version may include larger changes, but bug fixes will often not be included.
- I think that in most cases, bug fixes will cause the stable releases.
- They will not be released per branch or backported; they will simply be released.
- Bug fixes for previous versions will be applied to the latest beta version.
This means, if xx.yy.02+patched1 exists and there is a defect in xx.yy.01, a fix is applied to xx.yy.02+patched1 and yields xx.yy.02+patched2.
If the fix is required immediately, it is released as xx.yy.02 (with xx.yy.01+patched1).
This means, if xx.yy.02-patched1 exists and there is a defect in xx.yy.01, a fix is applied to xx.yy.02-patched1 and yields xx.yy.02-patched2.
If the fix is required immediately, it is released as xx.yy.02 (with xx.yy.01-patched1).
- This procedure remains unchanged from the current one.
- At the very least, I am using the latest beta.
- However, I will not be using a beta continuously for a week after it has been released. It is probably closer to an RC in nature.

View File

@@ -1,7 +1,6 @@
# For details and other explanations about this file refer to:
# https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/setup_own_server.md#traefik
version: "2.1"
services:
couchdb:
image: couchdb:latest

View File

@@ -0,0 +1,59 @@
# User Guide: Peer-to-Peer Synchronisation (2026 Edition)
Peer-to-Peer (P2P) synchronisation has evolved significantly. This guide covers the essential setup and the new features introduced in the 2026 updates.
## 1. Core Concept: Server-less Freedom
P2P synchronisation allows your devices to talk directly to each other using WebRTC. A central server is not required for data storage, ensuring maximum privacy and "freedom."
## 2. Setting Up via P2P Status Pane
You no longer need to navigate through complex menus. Simply open the **P2P Status** (via the ribbon icon or command palette) and click the **⚙ (Cog)** icon.
This opens the **P2P Setup** dialogue where you can configure the essentials:
- **Room ID:** A unique identifier for your synchronisation group.
- **Passphrase:** Your encryption key. Ensure all your devices use the exact same passphrase.
- **Device Name:** A recognisable name for the current device (e.g., `iphone-16`).
Once you have saved the settings, return to the **P2P Status Pane** and click the **Connect** button to join the network.
*Tip: You can also toggle **Auto Connect** in the setup dialogue to automatically join the network whenever Obsidian starts.*
## 3. Real-time Control
The status pane in the right sidebar provides granular control over your synchronisation:
- **Active P2P Remote (new):** P2P now has its own active remote selection, separate from the normal active remote for database replication. Use the combo box next to the cog icon to choose which P2P remote configuration is active for P2P features.
- **Create P2P Remote (new):** Use the **+** button to open the P2P setup dialogue and create a dedicated P2P remote configuration. This is recommended when no P2P active remote has been selected yet.
- **Selection required (new):** If no P2P active remote is selected, the pane asks for selection before P2P target-related changes are saved.
- **Signalling Status:** Shows if you are connected to the relay (🟢 Online).
- **Live-push (Broadcast):** Toggle "Broadcast changes" to notify other peers whenever you make an edit.
- **Replicate now (🔄):** Start immediate bidirectional replication with a visible peer (Pull, then Push).
- **Watch (🔔/🔕):** Enable "Watch" on specific peers to automatically pull changes when they broadcast. This creates a "LiveSync-like" experience.
- **Sync target (🔗/⛓️‍💥):** Mark specific peers as **sync targets**. Peers marked here will be included when you run the **"P2P: Sync with targets"** command (see section 5). Click the button next to a peer to toggle it on (🔗, highlighted) or off (⛓️‍💥). This setting is persisted in your configuration.
## 4. Replication Dialogue
If you want to synchronise with a specific peer manually, use the **Replication** command or button. This opens the **Replication Dialogue** listing available devices.
Inside the dialogue, the **Server Status** card at the top confirms you are still connected while performing the sync.
The status card now shows a stable **Room ID suffix** above **Peer ID**. The Room ID suffix is better for identifying your P2P group, while Peer ID may change between connections.
Two actions are available per peer:
- **Sync** — Starts a bidirectional synchronisation (Pull then Push) and keeps the dialogue open so you can monitor progress or sync with additional peers.
- **Start Sync & Close** — Starts the same bidirectional sync in the background and **immediately closes the dialogue**, so you can continue working without waiting.
## 5. Syncing with Registered Targets via Command Palette
You can now trigger a synchronisation with all your pre-registered target peers in one step, without opening any UI.
1. Open the **Command Palette** (`Ctrl/Cmd + P`).
2. Run **"P2P: Sync with targets"**.
This command synchronises with every peer whose **SYNC** toggle is enabled in the **Detected Peers** list. If no targets are registered, or if the P2P server is not running, the command will notify you accordingly.
*Tip: Pair this command with a hotkey for a quick, keyboard-driven sync workflow.*
## 6. Technical Improvements in 2026
- **Decoupled Architecture:** The UI is now strictly separated from the core logic, making the plug-in more stable across different platforms (Mobile, Desktop, and Web).
- **Svelte 5 UI:** The interface has been rebuilt for better responsiveness and clearer status indicators.
- **Security:** All data remains end-to-end encrypted. Even the signalling relay never sees your actual notes.

View File

@@ -2,7 +2,7 @@
[Japanese docs](./quick_setup_ja.md) - [Chinese docs](./quick_setup_cn.md).
The plugin has so many configuration options to deal with different circumstances. However, only a few settings are required in the normal cases. Therefore, `The Setup wizard` has been implemented to simplify the setup.
The plug-in has so many configuration options to deal with different circumstances. However, only a few settings are required in the normal cases. Therefore, `The Setup wizard` has been implemented to simplify the setup.
![](../images/quick_setup_1.png)
@@ -10,7 +10,7 @@ There are three methods to set up Self-hosted LiveSync.
1. [Using setup URIs](#1-using-setup-uris) *(Recommended)*
2. [Minimal setup](#2-minimal-setup)
3. [Full manually setup the and Enable on this dialogue](#3-manually-setup)
3. [Fully manual setup and enabling on this dialogue](#3-manually-setup)
## At the first device
@@ -24,7 +24,7 @@ There are three methods to set up Self-hosted LiveSync.
In this procedure, [this video](https://youtu.be/7sa_I1832Xc?t=146) may help us.
1. Click `Use` button (Or launch `Use the copied setup URI` from Command palette).
1. Click the `Use` button (or launch the `Use the copied setup URI (Formerly Open setup URI)` command from the command palette).
2. Paste the Setup URI into the dialogue
3. Type the passphrase of the Setup URI
4. Answer `yes` for `Importing LiveSync's conf, OK?`.
@@ -107,23 +107,27 @@ Note: If you are going to use Object Storage, you cannot select `LiveSync`.
Select any synchronisation methods we want to use and `Apply`. If database initialisation is required, it will be performed at this time. When `All done!` is displayed, we are ready to synchronise.
The dialogue of `Copy settings as a new setup URI` will be open automatically. Please input a passphrase to encrypt the new `Setup URI`. (This passphrase is to encrypt the setup URI, not the vault).
The dialogue of `Copy current settings as a new setup URI` will open automatically. Please input a passphrase to encrypt the new `Setup URI`. (This passphrase is to encrypt the setup URI, not the vault).
![](../images/quick_setup_10.png)
The Setup URI will be copied to the clipboard, please make a note(Not in Obsidian) of this.
>[!TIP]
We can copy this in any time by `Copy current settings as a new setup URI`.
We can copy this at any time by running the "Copy settings as a new setup URI" command from the command palette (or clicking the "Copy the current settings to a Setup URI" button in the settings UI).
### 3. Manually setup
It is strongly recommended to perform a "minimal set-up" first and set up the other contents after making sure has been synchronised.
However, if you have some specific reasons to configure it manually, please click the `Enable` button of `Enable LiveSync on this device as the set-up was completed manually`.
And, please copy the setup URI by `Copy current settings as a new setup URI` and make a note(Not in Obsidian) of this.
And, please copy the setup URI by running the "Copy settings as a new setup URI" command (or using the "Copy the current settings to a Setup URI" button) and make a note(Not in Obsidian) of this.
## At the subsequent device
After installing Self-hosted LiveSync on the first device, we should have a setup URI. **The first choice is to use it**. Please share it with the device you want to setup.
It is completely same as [Using setup URIs on the first device](#1-using-setup-uris). Please refer it.
> [!TIP]
> **Fast Setup (Simple Fetch)**
> In recent versions, when you import a Setup URI or trigger a Fetch All, the plug-in boots in scheduled fetch mode and runs a simplified **Fast Setup** process. This allows you to choose your sync strategy with a single dialogue and performs initial synchronisation in one step. Refer to the [Fast Setup Guide](./tips/fast-setup.md) for more details.

View File

@@ -1,6 +1,6 @@
# Quick setup
このプラグインには、いろいろな状況に対応するための非常に多くの設定オプションがあります。しかし、実際に使用する設定項目はそれほど多くはありません。そこで、初期設定を簡略化するために、「セットアップウィザード」を実装しています。
※なお、次のデバイスからは、`Copy setup URI``Open setup URI`を使ってセットアップしてください。
※なお、次のデバイスからは、`現在の設定をセットアップURIにコピー``セットアップURIで接続`を使ってセットアップしてください。
## Wizardの使い方
@@ -71,7 +71,8 @@ Fixボタンがなくなり、すべてチェックマークになれば完了
![](../images/quick_setup_9_1.png)
Presetsから、いずれかの同期方法を選び`Apply`を行うと、必要に応じてローカル・リモートのデータベースを初期化・構築します。
All done! と表示されれば完了です。自動的に、`Copy setup URI`が開き、`Setup URI`を暗号化するパスフレーズを聞かれます。
All done!」(日本語環境では「完了!」)と表示されれば完了です。自動的に、「現在の設定をセットアップURIにコピー」のダイアログが開き、Setup URIを暗号化するためのパスフレーズを求められますこのパスフレーズはSetup URIを暗号化するためのもので、Vault自体の暗号化キーではありません
パスフレーズを入力すると、クリップボードにSetup URIが保存されますので、これを2台目以降のデバイスに何らかの方法で転送してください。
![](../images/quick_setup_10.png)
@@ -79,10 +80,14 @@ All done! と表示されれば完了です。自動的に、`Copy setup URI`が
クリップボードにSetup URIが保存されますので、これを2台目以降のデバイスに何らかの方法で転送してください。
# 2台目以降の設定方法
2台目の端末にSelf-hosted LiveSyncをインストールしたあと、コマンドパレットから`Open setup URI`を選択し、転送したsetup URIを入力します。その後、パスフレーズを入力するとセットアップ用のウィザードが開きます。
2台目の端末にSelf-hosted LiveSyncをインストールしたあと、コマンドパレットから`Use the copied setup URI (Formerly Open setup URI)`を選択し、転送したsetup URIを入力します。その後、パスフレーズを入力するとセットアップ用のウィザードが開きます。
下記のように答えてください。
- `Importing LiveSync's conf, OK?``Yes`
- `How would you like to set it up?``Set it up as secondary or subsequent device`
これで設定が反映され、レプリケーションが開始されます。
これで設定が反映され、レプリケーションが開始されます。
> [!TIP]
> **ファストセットアップ (Fast Setup)**
> 近年のバージョンでは、セットアップURIの読み込みやデータの全取得Fetch Allを実行した際、より簡単に同期戦略を選択して即座に初期同期を完了できる **ファストセットアップ (Simple Fetch)** フローが利用できます。詳細は [ファストセットアップガイド](./tips/fast-setup_ja.md) をご参照ください。

View File

@@ -12,7 +12,7 @@ There are many settings in Self-hosted LiveSync. This document describes each se
| 🛰️ | [3. Remote Configuration](#3-remote-configuration) |
| 🔄 | [4. Sync Settings](#4-sync-settings) |
| 🚦 | [5. Selector (Advanced)](#5-selector-advanced) |
| 🔌 | [6. Customization sync (Advanced)](#6-customization-sync-advanced) |
| 🔌 | [6. Customisation sync (Advanced)](#6-customisation-sync-advanced) |
| 🧰 | [7. Hatch](#7-hatch) |
| 🔧 | [8. Advanced (Advanced)](#8-advanced-advanced) |
| 💪 | [9. Power users (Power User)](#9-power-users-power-user) |
@@ -68,7 +68,7 @@ Following panes will be shown when you enable this setting.
| Icon | Description |
| :--: | ------------------------------------------------------------------ |
| 🚦 | [5. Selector (Advanced)](#5-selector-advanced) |
| 🔌 | [6. Customization sync (Advanced)](#6-customization-sync-advanced) |
| 🔌 | [6. Customisation sync (Advanced)](#6-customisation-sync-advanced) |
| 🔧 | [8. Advanced (Advanced)](#8-advanced-advanced) |
#### Enable poweruser features
@@ -120,6 +120,18 @@ Setting key: showStatusOnStatusbar
We can show the status of synchronisation on the status bar. (Default: On)
#### Show status icon instead of file warnings banner
Setting key: hideFileWarningNotice
If enabled, the ⛔ icon will be shown inside the status instead of the file warnings banner. No details will be shown.
#### Network warning style
Setting key: networkWarningStyle
How to display network errors when the sync server is unreachable.
### 2. Logging
#### Show only notifications
@@ -138,11 +150,19 @@ Show verbose log. Please enable when you report the logs
### 1. Remote Server
Self-hosted LiveSync supports multiple remote connection profiles under **Remote Server** -> **Remote Databases**. This allows you to save and switch between multiple databases or bucket configurations in a single vault.
- ** Add new connection**: Create a new connection profile by launching the setup dialogue.
- **📥 Import connection**: Paste a connection string (e.g., `sls+https://...`, `sls+s3://...`, `sls+p2p://...`) to import a remote configuration profile.
- **🔧 Configure**: Open the setup dialogue to edit settings for the selected connection profile.
- **✅ Activate**: Select and activate this profile as the current active remote.
- **🗑️ Delete**: Remove this connection profile from the list.
#### Remote Type
Setting key: remoteType
Remote server type
The active remote server type. This is automatically projected to the legacy configuration when you activate a connection profile.
### 2. Notification
@@ -172,6 +192,14 @@ Setting key: usePathObfuscation
In default, the path of the file is not obfuscated to improve the performance. If you enable this, the path of the file will be obfuscated. This is useful when you want to hide the path of the file.
#### Encryption Algorithm
Setting key: E2EEAlgorithm
The encryption algorithm version used for end-to-end encryption.
- `v2` (V2: AES-256-GCM With HKDF): Recommended and default version.
- `forceV1` or `""` (V1: Legacy): Older legacy encryption. Only use this if you have an existing vault encrypted in the legacy format.
#### Use dynamic iteration count (Experimental)
Setting key: useDynamicIterationCount
@@ -192,30 +220,62 @@ Fetch necessary settings from already configured remote server.
### 5. Minio,S3,R2
These settings are configured within the S3/MinIO/R2 Setup dialogue when adding (``) or editing (`🔧`) an Object Storage connection profile.
#### Endpoint URL
Setting key: endpoint
The URL of the remote storage endpoint.
Note: Only Secure (HTTPS) connections can be used on Obsidian Mobile.
#### Access Key
Setting key: accessKey
The Access Key ID used for authentication.
#### Secret Key
Setting key: secretKey
The Secret Access Key used for authentication.
#### Region
Setting key: region
The storage region (e.g., `us-east-1`, or `auto` for Cloudflare R2).
#### Bucket Name
Setting key: bucket
The name of the bucket to store synchronised files.
#### Use Custom HTTP Handler
Setting key: useCustomRequestHandler
Enable this if your Object Storage doesn't support CORS
This option is labeled **Use internal API** in the setup dialogue. Enable this if your Object Storage does not support CORS. It uses Obsidian's internal API to communicate with the S3 server, which is not compliant with web standards but can bypass CORS restrictions. Note that this might break in future Obsidian versions.
#### File prefix on the bucket
Setting key: bucketPrefix
This option is labeled **Folder Prefix** in the setup dialogue. Effectively a directory. Should end with `/`. e.g., `vault-name/`. Leave blank to store data at the root of the bucket.
#### Enable forcePathStyle
Setting key: forcePathStyle
This option is labeled **Use Path-Style Access** in the setup dialogue. If enabled, the forcePathStyle option will be used for bucket operations.
#### Custom Headers
Setting key: bucketCustomHeaders
Custom HTTP headers to include in every request sent to the Object Storage bucket. Specify them in the format `Header-Name: Value`, with each header on a new line.
#### Test Connection
@@ -223,24 +283,82 @@ Enable this if your Object Storage doesn't support CORS
### 6. CouchDB
These settings are configured within the CouchDB Setup dialogue when adding (``) or editing (`🔧`) a CouchDB connection profile.
#### Server URI
Setting key: couchDB_URI
The URI of the CouchDB server.
Note: Only Secure (HTTPS) connections can be used on Obsidian Mobile. The URI must not end with a trailing slash.
#### Username
Setting key: couchDB_USER
username
The username used to authenticate with CouchDB.
#### Password
Setting key: couchDB_PASSWORD
password
The password used to authenticate with CouchDB.
#### Database Name
Setting key: couchDB_DBNAME
The name of the database.
Note: The database name cannot contain capital letters, spaces, or special characters other than `_$()+/-`, and cannot start with an underscore (`_`).
#### Use Request API to avoid inevitable CORS problem
Setting key: useRequestAPI
This option is labeled **Use Internal API** in the setup dialogue. If enabled, Obsidian's internal request API will be used to bypass CORS restrictions. This is a workaround that may not be compliant with web standards and is less secure. Note that this might break in future Obsidian versions.
#### Custom Headers
Setting key: couchDB_CustomHeaders
Custom HTTP headers to include in every request sent to the CouchDB server. Specify them in the format `Header-Name: Value`, with each header on a new line.
#### Use JWT Authentication
Setting key: useJWT
Enable JSON Web Token (JWT) authentication for CouchDB. This is an experimental feature and has not been thoroughly verified.
#### JWT Algorithm
Setting key: jwtAlgorithm
The algorithm used to sign the JWT. Supported algorithms: `HS256`, `HS512`, `ES256`, `ES512`.
#### JWT Expiration Duration (minutes)
Setting key: jwtExpDuration
Token expiration duration in minutes. Set to 0 to disable expiration.
#### JWT Key
Setting key: jwtKey
The secret key (for HS256/HS512) or the PKCS#8 PEM-formatted private key (for ES256/ES512) used to sign the JWT.
#### JWT Key ID (kid)
Setting key: jwtKid
The Key ID (`kid`) header parameter included in the JWT.
#### JWT Subject (sub)
Setting key: jwtSub
The subject (`sub`) claim of the JWT, which should match your CouchDB username.
#### Test Database Connection
Open database connection. If the remote database is not found and you have permission to create a database, the database will be created.
@@ -251,26 +369,100 @@ Checks and fixes any potential issues with the database config.
#### Apply Settings
### 7. Peer-to-Peer (P2P) Synchronisation
#### Enable P2P Synchronisation
Setting key: P2P_Enabled
Enable direct peer-to-peer synchronisation via WebRTC.
#### Relay URL
Setting key: P2P_relays
The WebSocket relay server URL(s) used for coordinating P2P connections via WebRTC. Multiple URLs can be separated by commas.
#### Group ID
Setting key: P2P_roomID
The room ID or Group ID used to identify your group of synchronising devices. All devices you wish to synchronise must use the same Group ID. You can enter any custom string or generate a random Group ID.
#### Passphrase
Setting key: P2P_passphrase
The password or passphrase used to authenticate and encrypt P2P communication. All devices must use the same passphrase.
#### Device Peer ID
Setting key: P2P_DevicePeerName
The peer name or identifier of this device in the P2P network. This should be unique within your group of devices.
#### Automatically start P2P connection on launch
Setting key: P2P_AutoStart
This option is labeled **Auto Start P2P Connection** in the setup dialogue. If enabled, the P2P connection will start automatically when the plug-in launches.
#### Automatically broadcast changes to connected peers
Setting key: P2P_AutoBroadcast
This option is labeled **Auto Broadcast Changes** in the setup dialogue. If enabled, changes will be automatically broadcasted to connected peers, requesting them to fetch the changes.
#### TURN Server URLs (comma-separated)
Setting key: P2P_turnServers
A comma-separated list of TURN/STUN server URLs. Used to relay P2P connections when direct WebRTC connection fails due to strict NAT or firewalls. In most cases, these can be left blank.
#### TURN Username
Setting key: P2P_turnUsername
The username for authentication with the TURN server.
#### TURN Credential
Setting key: P2P_turnCredential
The password or credential for authentication with the TURN server.
## 4. Sync Settings
### 1. Synchronization Preset
### 1. Synchronisation Preset
#### Presets
Setting key: preset
Apply preset configuration
### 2. Synchronization Method
### 2. Synchronisation Method
#### Sync Mode
Setting key: syncMode
The trigger mechanism for synchronisation.
- **LiveSync** (`LIVESYNC`): Real-time, continuous, bidirectional synchronisation.
Note: This requires a CouchDB or WebRTC P2P remote server. It is not supported for S3-compatible Object Storage.
- **Periodic Sync** (`PERIODIC`): Synchronisation is performed at regular intervals specified by the **Periodic Sync interval** setting.
- **On Events** (`ONEVENTS`): Synchronisation is triggered by specific events (such as save, file open, or startup) configured via the toggles below.
#### Periodic Sync interval
Setting key: periodicReplicationInterval
Interval (sec)
#### Minimum interval for syncing
Setting key: syncMinimumInterval
The minimum interval for automatic synchronisation on event.
#### Sync on Save
Setting key: syncOnSave
@@ -323,7 +515,7 @@ Move remotely deleted files to the trash, instead of deleting.
#### Keep empty folder
Setting key: doNotDeleteFolder
Should we keep folders that don't have any files inside?
Should we keep folders that do not have any files inside?
### 5. Conflict resolution (Advanced)
@@ -360,7 +552,7 @@ Setting key: notifyAllSettingSyncFile
### 7. Hidden Files (Advanced)
#### Hidden file synchronization
#### Hidden file synchronisation
#### Enable Hidden files sync
@@ -373,6 +565,12 @@ Setting key: syncInternalFilesBeforeReplication
Setting key: syncInternalFilesInterval
Seconds, 0 to disable
#### Suppress notification of hidden files change
Setting key: suppressNotifyHiddenFilesChange
If enabled, the notification of hidden files change will be suppressed.
## 5. Selector (Advanced)
### 1. Normal Files
@@ -406,42 +604,42 @@ Comma separated `.gitignore, .dockerignore`
#### Add default patterns
## 6. Customization sync (Advanced)
## 6. Customisation sync (Advanced)
### 1. Customization Sync
### 1. Customisation Sync
#### Device name
Setting key: deviceAndVaultName
Unique name between all synchronized devices. To edit this setting, please disable customization sync once.
Unique name between all synchronised devices. To edit this setting, please disable customisation sync once.
#### Per-file-saved customization sync
#### Per-file-saved customisation sync
Setting key: usePluginSyncV2
If enabled per-filed efficient customization sync will be used. We need a small migration when enabling this. And all devices should be updated to v0.23.18. Once we enabled this, we lost a compatibility with old versions.
If enabled, per-file efficient customisation sync will be used. We need a small migration when enabling this. And all devices should be updated to v0.23.18. Once we enable this, we lose compatibility with old versions.
#### Enable customization sync
#### Enable customisation sync
Setting key: usePluginSync
#### Scan customization automatically
#### Scan customisation automatically
Setting key: autoSweepPlugins
Scan customization before replicating.
Scan customisation before replicating.
#### Scan customization periodically
#### Scan customisation periodically
Setting key: autoSweepPluginsPeriodic
Scan customization every 1 minute.
Scan customisation every 1 minute.
#### Notify customized
#### Notify customised
Setting key: notifyPluginOrSettingUpdated
Notify when other device has newly customized.
Notify when another device has newly customised.
#### Open
Open the dialog
Open the dialogue
## 7. Hatch
@@ -456,14 +654,18 @@ Warning! This will have a serious impact on performance. And the logs will not b
### 2. Scram Switches
Emergency controls to suspend synchronisation processes in order to prevent database corruption. If a critical mismatch or sync error occurs, the plug-in may automatically enter a Scram state and suspend operations.
#### Suspend file watching
Setting key: suspendFileWatching
Stop watching for file changes.
Stop watching for local file changes.
#### Suspend database reflecting
Setting key: suspendParseReplicationResult
Stop reflecting database changes to storage files.
### 3. Recovery and Repair
@@ -486,7 +688,7 @@ Compare the content of files between on local database and storage. If not match
#### Back to non-configured
#### Delete all customization sync data
#### Delete all customisation sync data
## 8. Advanced (Advanced)
@@ -507,6 +709,12 @@ Setting key: hashCacheMaxAmount
Setting key: customChunkSize
#### Chunk Splitter
Setting key: chunkSplitterVersion
Select the chunk splitter version; V3 is the most efficient. If you experience issues, please choose Default or Legacy.
#### Use splitting-limit-capped chunk splitter
Setting key: enableChunkSplitterV2
@@ -532,6 +740,12 @@ Setting key: concurrencyOfReadChunksOnline
Setting key: minimumIntervalOfReadChunksOnline
#### Maximum size of chunks to send in one request
Setting key: sendChunksBulkMaxSize
Limit the maximum size of chunks to send in a single bulk request (MB).
## 9. Power users (Power User)
### 1. Remote Database Tweak
@@ -639,7 +853,7 @@ If this enabled, All files are handled as case-Sensitive (Previous behaviour).
### 4. Compatibility (Internal API Usage)
#### Scan changes on customization sync
#### Scan changes on customisation sync
Setting key: watchInternalFileChanges
Do not use internal API
@@ -664,7 +878,13 @@ Setting key: doNotSuspendOnFetching
#### Keep empty folder
Setting key: doNotDeleteFolder
Should we keep folders that don't have any files inside?
Should we keep folders that do not have any files inside?
#### Process files even if seems to be corrupted
Setting key: processSizeMismatchedFiles
Enable this setting to process files with size mismatches, which can sometimes be created by certain external APIs or integrations.
### 7. Edge case addressing (Processing)
@@ -684,17 +904,25 @@ If enabled, the file under 1kb will be processed in the UI thread.
Setting key: disableCheckingConfigMismatch
### 9. Remediation
#### Maximum file modification time for reflected file events
Setting key: maxMTimeForReflectEvents
Files with modification times greater than this value (in seconds since the Unix epoch) will not have their events reflected. Set to 0 to disable this limit.
## 11. Maintenance
### 1. Scram!
#### Lock Server
Lock the remote server to prevent synchronization with other devices.
Lock the remote server to prevent synchronisation with other devices.
#### Emergency restart
Disables all synchronization and restart.
Disables all synchronisation and restart.
### 2. Syncing
@@ -712,17 +940,13 @@ Initialise journal sent history. On the next sync, every item except this device
### 3. Rebuilding Operations (Local)
#### Fetch from remote
#### Reset Synchronisation on This Device
Restore or reconstruct local database from remote.
#### Fetch rebuilt DB (Save local documents before)
Restore or reconstruct local database from remote database but use local chunks.
### 4. Total Overhaul
#### Rebuild everything
#### Overwrite Server Data with This Device's Files
Rebuild local and remote database with local files.
@@ -752,7 +976,7 @@ Delete all data on the remote server.
#### Run database cleanup
Attempt to shrink the database by deleting unused chunks. This may not work consistently. Use the 'Rebuild everything' under Total Overhaul.
Attempt to shrink the database by deleting unused chunks. This may not work consistently. Use the 'Overwrite Server Data with This Device's Files' under Reset Synchronisation information.
### 7. Reset

View File

@@ -3,23 +3,133 @@
# このプラグインの設定項目
## Remote Database Configurations
同期先のデータベース設定を行います。何らかの同期が有効になっている場合は編集できないため、同期を解除してから行ってください
同期先のデータベース設定Remote Serverを行います。
### URI
CouchDBのURIを入力します。Cloudantの場合は「External Endpoint(preferred)」になります。
**スラッシュで終わってはいけません。**
こちらにデータベース名を含めてもかまいません。
現在のバージョンでは、複数のリモート接続設定接続プロファイルを登録・管理し、切り替えて使用することが可能です「Remote Databases」リスト
### Username
ユーザー名を入力します。このユーザーは管理者権限があることが望ましいです。
- ** 新規接続を追加 (Add new connection)**: 新しい接続設定を作成し、各セットアップダイアログを起動します。
- **📥 接続をインポート (Import connection)**: 接続文字列(`sls+https://...``sls+s3://...``sls+p2p://...`など)を貼り付けてインポートします。
- **🔧 設定 (Configure)**: セットアップダイアログを開き、選択した接続プロファイルの設定を編集します。
- **✅ 有効化 (Activate)**: 選択したプロファイルをアクティブな同期先として有効化します。
- **🗑️ 削除 (Delete)**: 接続プロファイルを一覧から削除します。
### Password
パスワードを入力します。
これらの接続プロファイルを追加・編集する際、選択したデータベースの種類CouchDB、S3互換オブジェクトストレージ、P2Pなどに応じたセットアップダイアログが開きます。
### Database Name
同期するデータベース名を入力します。
⚠️存在しない場合は、テストや接続を行った際、自動的に作成されます[^1]。
[^1]:権限がない場合は自動作成には失敗します。
何らかの同期が有効になっている場合は編集できないため、同期を解除してから行ってください。
### CouchDB の設定
CouchDBの各設定項目は、接続プロファイルを追加 () または設定 (🔧) する際に開く **CouchDB セットアップダイアログ** 内で設定します。
#### URI
設定キー: couchDB_URI
CouchDBの接続先URIです。ダイアログ内では **URL** と表記されます。Cloudantの場合は「External Endpoint (preferred)」になります。
注意: Obsidian Mobileではセキュア接続 (HTTPS) のみが使用可能です。また、末尾にスラッシュ(`/`)を付けてはいけません。
#### Username
設定キー: couchDB_USER
CouchDBのログインユーザー名です。ダイアログ内では **Username** と表記されます。このユーザーには管理者権限があることが望ましいです。
#### Password
設定キー: couchDB_PASSWORD
CouchDBのログインパスワードです。ダイアログ内では **Password** と表記されます。
#### Database Name
設定キー: couchDB_DBNAME
同期先のデータベース名です。ダイアログ内では **Database Name** と表記されます。
注意: データベース名には大文字、スペース、および一部の特殊文字(`_$()+/-` 以外)は使用できません。また、アンダースコア(`_`)から始めることはできません。存在しない場合は、接続テスト時または設定適用時に自動作成されます(作成権限が必要です)。
#### CORS回避のためにRequest APIを使用する
設定キー: useRequestAPI
この項目はセットアップダイアログ内では **Use Internal API** と表記されます。有効な場合、不可避なCORS問題を回避するためにObsidianの内部Request APIを使用します。これはWeb標準に準拠していない回避策であり、すべての環境での動作を保証するものではありません。安全性が低下する可能性がある点にご注意ください。将来のObsidianのアップデートによって動作しなくなる可能性があります。
#### カスタムヘッダー
設定キー: couchDB_CustomHeaders
CouchDBサーバーに送信するすべてのリクエストに含めるカスタムHTTPヘッダーを設定します。ダイアログ内では **Custom Headers** と表記されます。`ヘッダー名: 値` の形式で、1行に1つずつ入力してください。
#### JWT認証の使用 (実験的機能)
設定キー: useJWT
CouchDBでのJSON Web Token (JWT) 認証を有効にします。ダイアログ内では **Use JWT Authentication** と表記されます。十分に検証されていない実験的機能であるため、ご注意ください。
#### JWTアルゴリズム
設定キー: jwtAlgorithm
JWTの署名に使用するアルゴリズムを選択します。ダイアログ内では **JWT Algorithm** と表記されます。対応アルゴリズム: `HS256`, `HS512`, `ES256`, `ES512`
#### JWT有効期限 (分)
設定キー: jwtExpDuration
トークンの有効期限を分単位で指定します。ダイアログ内では **JWT Expiration Duration (minutes)** と表記されます。`0` を指定すると有効期限は無効になります。
#### JWTキー
設定キー: jwtKey
JWTの署名に使用する秘密鍵またはプライベートキーを指定します。ダイアログ内では **JWT Key** と表記されます。`HS256/HS512` の場合は共通鍵を、`ES256/ES512` の場合は pkcs8 PEM形式の秘密鍵を入力してください。
#### JWTキーID (kid)
設定キー: jwtKid
JWTヘッダーに含めるキーIDを指定します。ダイアログ内では **JWT Key ID (kid)** と表記されます。
#### JWTサブジェクト (sub)
設定キー: jwtSub
JWTのサブジェクト (CouchDBユーザー名) を指定します。ダイアログ内では **JWT Subject (sub)** と表記されます。
### Object Storage (Minio, S3, R2) の設定
Object Storageの各設定項目は、接続プロファイルを追加 () または設定 (🔧) する際に開く **S3/MinIO/R2 セットアップダイアログ** 内で設定します。
#### エンドポイントURL
設定キー: endpoint
S3互換ストレージのエンドポイントURLです。ダイアログ内では **Endpoint URL** と表記されます。
注意: Obsidian Mobileではセキュア接続 (HTTPS) のみが使用可能です。
#### アクセスキー ID
設定キー: accessKey
認証に使用するアクセスキーIDです。ダイアログ内では **Access Key ID** と表記されます。
#### シークレットアクセスキー
設定キー: secretKey
認証に使用するシークレットアクセスキーです。ダイアログ内では **Secret Access Key** と表記されます。
#### リージョン
設定キー: region
ストレージのリージョンを指定します(例: `us-east-1`、Cloudflare R2の場合は `auto`)。ダイアログ内では **Region** と表記されます。
#### バケット名
設定キー: bucket
同期データを保存するバケット名です。ダイアログ内では **Bucket Name** と表記されます。
#### カスタムHTTPハンドラーを使用する
設定キー: useCustomRequestHandler
この項目はセットアップダイアログ内では **Use internal API** と表記されます。オブジェクトストレージがCORSをサポートしていない場合に有効にします。Obsidianの内部APIを使用してS3サーバーと通信することでCORS制約を回避します。Web標準には準拠していないため、将来のObsidianのアップデートによって動作しなくなる可能性があります。
#### バケット内のファイルプレフィックス
設定キー: bucketPrefix
この項目はセットアップダイアログ内では **Folder Prefix** と表記されます。実質的なディレクトリ指定です。末尾は `/` である必要があります(例:`vault-name/`)。バケットのルートに保存する場合は空欄のままにしてください。
#### forcePathStyleを有効にする
設定キー: forcePathStyle
この項目はセットアップダイアログ内では **Use Path-Style Access** と表記されます。有効な場合、バケット操作でforcePathStyleオプションを使用します。
#### カスタムヘッダー
設定キー: bucketCustomHeaders
オブジェクトストレージバケットに送信するすべてのリクエストに含めるカスタムHTTPヘッダーを設定します。ダイアログ内では **Custom Headers** と表記されます。`ヘッダー名: 値` の形式で、1行に1つずつ入力してください。
@@ -30,6 +140,18 @@ CouchDBのURIを入力します。Cloudantの場合は「External Endpoint(prefe
### Passphrase
暗号化を行う際に使用するパスフレーズです。充分に長いものを使用してください。
### パスの難読化
設定キー: usePathObfuscation
ダイアログ内では **Obfuscate Properties** と表記されます。有効な場合、リモートサーバー上でのファイルパスやフォルダ名を難読化(暗号化)します。これによりプライバシーが向上しますが、パフォーマンスがわずかに低下する可能性があります。
### 暗号化アルゴリズム
設定キー: E2EEAlgorithm
ダイアログ内では **Encryption Algorithm** と表記されます。エンドツーエンド暗号化に使用する暗号化アルゴリズムのバージョンを選択します。
- `v2` (V2: AES-256-GCM With HKDF): 推奨されるデフォルトのバージョンです。
- `forceV1` または `""` (V1: Legacy): レガシーな暗号化バージョンです。古いバージョンで暗号化された既存の保管庫Vaultを同期する場合にのみ使用してください。
### Apply
End to End 暗号化を行うに当たって、異なるパスフレーズで暗号化された同一の内容を入手されることは避けるべきです。また、Self-hosted LiveSyncはコンテンツのcrc32を重複回避に使用しているため、その点でも攻撃が有効になってしまいます。
@@ -53,12 +175,66 @@ End to End 暗号化を行うに当たって、異なるパスフレーズで暗
どちらのオペレーションも、実行するとすべての同期設定が無効化されます。
### Test Database connection
上記の設定でデータベースに接続できるか確認します。
### Check database configuration
ここから直接CouchDBの設定を確認・変更できます。
### Peer-to-Peer (P2P) 同期の設定
#### P2P同期を有効にする
設定キー: P2P_Enabled
WebRTCを介したデバイス間での直接的なP2P同期を有効にします。ダイアログ内では **Enabled** と表記されます。
#### リレーサーバーのURL
設定キー: P2P_relays
WebRTCによるP2P接続を仲介・調整するためのWebSocketリレーサーバーのURLを指定します。ダイアログ内では **Relay URL** と表記されます。複数のURLを指定する場合はカンマで区切ります。ダイアログ内のボタンをクリックすると、デフォルトのリレーサーバーを設定できます。
#### グループID
設定キー: P2P_roomID
同期するデバイス群を識別するためのルームIDまたはグループIDを指定します。ダイアログ内では **Group ID** と表記されます。同期させたいすべてのデバイスで同じグループIDを指定する必要があります。任意のカスタム文字列を入力するか、ランダム生成ボタンで生成できます。
#### パスフレーズ
設定キー: P2P_passphrase
P2P通信の認証および暗号化に使用するパスワードパスフレーズを指定します。ダイアログ内では **Passphrase** と表記されます。同期するすべてのデバイスで同じパスフレーズを指定する必要があります。
#### デバイス名
設定キー: P2P_DevicePeerName
P2Pネットワーク上でこのデバイスを識別するための名前を指定します。ダイアログ内では **Device Peer ID** と表記されます。グループ内のデバイス間で重複しない一意の値を設定してください。
#### 起動時のP2P自動接続開始
設定キー: P2P_AutoStart
有効な場合、プラグインの起動時に自動的にP2P接続を開始します。ダイアログ内では **Auto Start P2P Connection** と表記されます。
#### 接続済みピアへの変更の自動ブロードキャスト
設定キー: P2P_AutoBroadcast
有効な場合、ローカルでの変更が接続済みのピアに自動的にブロードキャストされます。ダイアログ内では **Auto Broadcast Changes** と表記されます。通知されたピアは変更の取得を開始します。
#### TURNサーバーのURL (カンマ区切り)
設定キー: P2P_turnServers
ダイアログ内では **TURN Server URLs (comma-separated)** と表記されます。厳しいNATやファイアウォールがある環境で、WebRTCの直接接続が確立できない場合にP2P接続を中継するためのTURN/STUNサーバーのURLをカンマ区切りで指定します。通常は空欄のままで問題ありません。
#### TURNユーザー名
設定キー: P2P_turnUsername
TURNサーバーでの認証に使用するユーザー名を設定します。ダイアログ内では **TURN Username** と表記されます。
#### TURNパスワード
設定キー: P2P_turnCredential
TURNサーバーでの認証に使用するパスワードクレデンシャルを設定します。ダイアログ内では **TURN Credential** と表記されます。
## Local Database Configurations
端末内に作成されるデータベースの設定です。
@@ -71,7 +247,8 @@ End to End 暗号化を行うに当たって、異なるパスフレーズで暗
このオプションはLiveSyncと同時には使用できません。
### minimum chunk size と LongLine threshold
チャンクの分割についての設定です。
チャンクの分割についての設定です。※現在これらの項目はUIから直接設定することはできませんデフォルト値で自動処理されます
Self-hosted LiveSyncは一つのチャンクのサイズを最低minimum chunk size文字確保した上で、できるだけ効率的に同期できるよう、ートを分割してチャンクを作成します。
これは、同期を行う際に、一定の文字数で分割した場合、先頭の方を編集すると、その後の分割位置がすべてずれ、結果としてほぼまるごとのファイルのファイル送受信を行うことになっていた問題を避けるために実装されました。
具体的には、先頭から順に直近の下記の箇所を検索し、一番長く切れたものを一つのチャンクとします。
@@ -88,6 +265,11 @@ Self-hosted LiveSyncは一つのチャンクのサイズを最低minimum chunk s
改行文字と#を除き、すべて●に置換しても、アルゴリズムは有効に働きます。
デフォルトは20文字と、250文字です。
### チャンクスプリッター
設定キー: chunkSplitterVersion
チャンク分割アルゴリズムを選択します。V3が最も効率的です。問題が発生した場合はDefaultまたはLegacyに設定してください。
## General Settings
一般的な設定です。
@@ -97,18 +279,35 @@ Self-hosted LiveSyncは一つのチャンクのサイズを最低minimum chunk s
### Vervose log
詳細なログをログに出力します。
### ファイル警告バナーの代わりにステータスアイコンを表示
設定キー: hideFileWarningNotice
有効な場合、ファイル警告バナーの代わりにステータス表示内に ⛔ アイコンが表示されます(詳細情報は非表示になります)。
### ネットワーク警告のスタイル
設定キー: networkWarningStyle
同期サーバーに接続できない場合のネットワークエラーの表示方法。
## Sync setting
同期に関する設定です。
### LiveSync
LiveSyncを行います。
他の同期方法では、同期の順序が「バージョン確認を行い、ロックが行われていないか確認した後、リモートの変更を受信した後、デバイスの変更を送信する」という挙動になります。
### 同期モード (Sync Mode)
設定キー: syncMode
### Periodic Sync
定期的に同期を行います。
同期処理を実行するトリガーとなる条件を設定します。
- **LiveSync** (`LIVESYNC`): リアルタイムかつ継続的な双方向同期を行います。
注意: このモードには CouchDB または WebRTC P2P リモートサーバーが必要です。S3互換オブジェクトストレージではサポートされていません。
- **Periodic Sync** (`PERIODIC`): **Periodic Sync Interval** で指定した一定の間隔ごとに同期処理を実行します。
- **On Events** (`ONEVENTS`): ファイルの保存、ファイルを開く、起動時など、特定のイベントが発生した際に同期をトリガーします(詳細は下部の設定スイッチで制御します)。
### Periodic Sync Interval
定期的に同期を行う場合の間隔です。
定期的に同期を行う場合の間隔(秒単位)です。
### 同期の最小間隔
設定キー: syncMinimumInterval
イベント時の自動同期の最小間隔(ミリ秒)。
### Sync on Save
ファイルが保存されたときに同期を行います。
@@ -146,6 +345,11 @@ Self-hosted LiveSyncは通常、フォルダ内のファイルがすべて削除
- Scan hidden files periodicaly.
このオプションを有効にすると、n秒おきに隠しファイルをスキャンします。
#### 非表示ファイルの変更通知を抑制
設定キー: suppressNotifyHiddenFilesChange
有効な場合、非表示ファイルの変更に関する通知を抑制します。
隠しファイルは能動的に検出されないため、スキャンが必要です。
スキャンでは、ファイルと共にファイルの変更時刻を保存します。もしファイルが消された場合は、その事実も保存します。このファイルを記録したエントリーがレプリケーションされた際、ストレージよりも新しい場合はストレージに反映されます。
@@ -176,6 +380,45 @@ Self-hosted LiveSyncはPouchDBを使用し、リモートと[このプロトコ
### Batch limit
一度に処理するBatchの数です。デフォルトは40です。
### 1回のリクエストで送信するチャンクの最大サイズ
設定キー: sendChunksBulkMaxSize
メガバイトMB単位で指定します。
## Customisation Sync (カスタマイズ同期)
プラグイン、ホットキー、テーマ、スニペットなどのObsidianのカスタマイズ設定を同期する機能です以前は **Plugin Sync** と呼ばれていました)。
### デバイス名 (Device name)
設定キー: deviceAndVaultName
同期するすべてのデバイス間で一意となるデバイス名です。この設定を編集するには、一度カスタマイズ同期を無効にする必要があります。
### ファイル保存ごとのカスタマイズ同期 (Per-file-saved customisation sync)
設定キー: usePluginSyncV2
有効な場合、ファイルごとの効率的なカスタマイズ同期が使用されます。有効にする際には簡単な移行作業が必要であり、すべてのデバイスを v0.23.18 以降にアップデートする必要があります。この機能を有効にすると、古いバージョンとの互換性が失われます。
### カスタマイズ同期を有効にする (Enable customisation sync)
設定キー: usePluginSync
テーマ、スニペット、ホットキー、プラグイン設定などの同期を有効にします。
注意: 安全上の理由から、この機能を使用するにはエンドツーエンド暗号化End-to-End Encryptionが有効になっている必要があります。
### カスタマイズの自動スキャン (Scan customisation automatically)
設定キー: autoSweepPlugins
レプリケーション(同期処理)を実行する前に、カスタマイズ設定の変更をスキャンします。
### 定期的なカスタマイズのスキャン (Scan customisation periodically)
設定キー: autoSweepPluginsPeriodic
1分ごとにカスタマイズ設定の変更を定期的にスキャンします。
### カスタマイズ更新の通知 (Notify customised)
設定キー: notifyPluginOrSettingUpdated
他のデバイスで新しくカスタマイズ設定が更新されたときに通知を表示します。
## Miscellaneous
その他の設定です
### Show status inside editor
@@ -195,8 +438,8 @@ Self-hosted LiveSyncはPouchDBを使用し、リモートと[このプロトコ
![CorruptedData](../images/lock_pattern1.png)
データベースがロックされていて、端末が「解決済み」とマークされていない場合、警告が表示されます。
他のデバイスで、End to End暗号化を有効にしたか、Drop Historyを行った等、他の端末がそのまま同期を行ってはいない状態に陥った場合表示されます。
暗号化を有効化した場合は、パスフレーズを設定してApply and recieve、Drop Historyを行った場合は、Drop and recieveを行うと自動的に解除されます。
手動でこのロックを解除する場合は「mark this device as resolved」をクリックしてください。
暗号化を有効化した場合は、パスフレーズを設定して「このデバイスの同期状態をリセット」、または「このデバイスのファイルでサーバーデータを上書き」を行うと自動的に解除されます。
手動でこのロックを解除する場合は「I've made a backup, mark this device 'resolved'」をクリックしてください。
- パターン2
![CorruptedData](../images/lock_pattern2.png)
@@ -207,18 +450,52 @@ Self-hosted LiveSyncはPouchDBを使用し、リモートと[このプロトコ
### Verify and repair all files
Vault内のファイルを全て読み込み直し、もし差分があったり、データベースから正常に読み込めなかったものに関して、データベースに反映します。
- Drop and send
デバイスとリモートのデータベースを破棄し、ロックしてからデバイスのファイルでデータベースを構築後、リモートに上書きします。
- Drop and receive
デバイスのデータベースを破棄した後、リモートから、操作しているデバイスに関してロックを解除し、データを受信して再構築します。
- このデバイスの同期状態をリセット (Reset Synchronisation on This Device)
ローカルのデータベースを破棄し、リモートのデータから再構築します。
- このデバイスのファイルでサーバーデータを上書き (Overwrite Server Data with This Device's Files)
ローカルおよびリモートのデータベースをこのデバイス上のファイルで再構築(上書き)します。
### Lock remote database
リモートのデータベースをロックし、他の端末で同期を行おうとしてもエラーとともに同期がキャンセルされるように設定します。これは、データベースの再構築を行った場合、自動的に設定されるものと同じものです。
万が一同期に不具合が発生していて、使用しているデバイスのデータ+サーバーのデータを保護する場合などに、緊急避難的に使用してください。
### Suspend file watching
ファイルの更新の監視を止めます。
### Scram スイッチ (Scram Switches)
データベースの破損や予期しないデータ喪失を防ぐために、同期処理を緊急停止するためのスイッチです。重大な設定不一致や同期エラーが発生した場合、プラグインは自動的に Scram 状態に移行し、同期動作を一時停止することがあります。
#### ファイルの更新監視を一時停止 (Suspend file watching)
設定キー: suspendFileWatching
ローカルファイル変更の監視と検知を停止します。
#### データベース反映を一時停止 (Suspend database reflecting)
設定キー: suspendParseReplicationResult
データベースでの変更をストレージファイルVault内のファイルへ書き戻す処理を停止します。
### 互換性(メタデータ)(Compatibility (Metadata))
#### 削除済みファイルのメタデータを保持しない (Do not keep metadata of deleted files.)
設定キー: deleteMetadataOfDeletedFiles
ファイルを削除した際に、そのファイルの同期履歴メタデータも即座にデータベースから削除し、保持しないようにします。
#### 削除済みデータのメタデータをクリーンナップする (Delete old metadata of deleted files on start-up)
設定キー: automaticallyDeleteMetadataOfDeletedFiles
ファイルを削除した際のメタデータを保持する期間(日数)を設定します。指定した日数を経過した古い削除済みファイルのメタデータは、プラグイン起動時にデータベースから自動的に削除(クリーンナップ)されます。`0` を指定すると自動削除は無効になります。
### 破損している可能性があるファイルも処理する
設定キー: processSizeMismatchedFiles
サイズ不一致のあるファイルを処理します。特定のAPIや外部連携によって作成されたファイルを同期する際に役立ちます。
### Remediation
#### イベント反映時の最大ファイル更新日時
設定キー: maxMTimeForReflectEvents
この値Unixエポックからの秒数より新しい更新日時を持つファイルについては、イベントの反映を無視します。0を指定すると制限が無効になります。
### Corrupted data
![CorruptedData](../images/corrupted_data.png)

View File

@@ -13,7 +13,7 @@ In these instructions, create IBM Cloudant Instance for trial.
1. You can choose "Lite plan" for free.
![step 3](../instruction_images/cloudant_3.png)
1. Select Multitenant(it's the default) and the region as you like.
1. Select Multitenant (it is the default) and the region as you like.
![step 4](../instruction_images/cloudant_4.png)
1. Be sure to select "IAM and Legacy credentials" for "Authentication Method".
@@ -28,20 +28,20 @@ In these instructions, create IBM Cloudant Instance for trial.
1. When all of the above steps have been done, open "Resource list" on the left pane. you can see the Cloudant instance in the "Service and software". Click it.
![step 8](../instruction_images/cloudant_8.png)
1. In resource details, there's information to connect from Self-hosted LiveSync.
Copy the "External Endpoint(preferred)" address. <sup>(\*1)</sup>. We use this address later, with the database name.
1. In resource details, there is information to connect from Self-hosted LiveSync.
Copy the "External Endpoint (preferred)" address. <sup>(\*1)</sup>. We use this address later, with the database name.
![step 9](../instruction_images/cloudant_9.png)
## Database setup
1. Hit the "Launch Dashboard" button, Cloudant dashboard will be shown.
Yes, it's almost CouchDB's fauxton.
Yes, it is almost CouchDB's fauxton.
![step 1](../instruction_images/couchdb_1.png)
1. First, you have to enable the CORS option.
Hit the Account menu and open the "CORS" tab.
Initially, "Origin Domains" is set to "Restrict to specific domains"., so set to "All domains(\*)"
_NOTE: of course We want to set "app://obsidian.md" but it's not acceptable on Cloudant._
_NOTE: of course We want to set "app://obsidian.md" but it is not acceptable on Cloudant._
![step 2](../instruction_images/couchdb_2.png)
1. Next, Open the "Databases" tab and hit the "Create Database" button.
@@ -55,10 +55,10 @@ In these instructions, create IBM Cloudant Instance for trial.
### Credentials Setup
1. Back into IBM Cloud, Open the "Service credentials". You'll get an empty list, hit the "New credential" button.
1. Back into IBM Cloud, Open the "Service credentials". You will get an empty list, hit the "New credential" button.
![step 1](../instruction_images/credentials_1.png)
1. The dialog to create a credential will be shown.
1. The dialogue to create a credential will be shown.
type any name or leave it default, hit the "Add" button.
![step 2](../instruction_images/credentials_2.png)
_NOTE: This "name" is not related to your username that uses in Self-hosted LiveSync._
@@ -68,14 +68,14 @@ In these instructions, create IBM Cloudant Instance for trial.
![step 3](../instruction_images/credentials_3.png)
The username and password pair is inside this JSON.
"username" and "password" are so.
follow the figure, it's
follow the figure, it is
"apikey-v2-2unu15184f7o8emr90xlqgkm2ncwhbltml6tgnjl9sd5"<sup>(\*3)</sup> and "c2c11651d75497fa3d3c486e4c8bdf27"<sup>(\*4)</sup>
## Self-hosted LiveSync settings
![Setting](../images/remote_db_setting.png)
The Setting should be as below:
The settings should be as follows:
| Items | Value | example |
| ------------- | ----- | ----------------------------------------------------------------- |

View File

@@ -230,7 +230,6 @@ And, be sure to check the server log and be careful of malicious access.
If you are using Traefik, this [docker-compose.yml](https://github.com/vrtmrz/obsidian-livesync/blob/main/docker-compose.traefik.yml) file (also pasted below) has all the right CORS parameters set. It assumes you have an external network called `proxy`.
```yaml
version: "2.1"
services:
couchdb:
image: couchdb:latest

View File

@@ -71,7 +71,6 @@ obsidian-livesync
可以参照以下内容编辑 `docker-compose.yml`:
```yaml
version: "2.1"
services:
couchdb:
image: couchdb

View File

@@ -1,8 +1,8 @@
# Designed architecture
## How does this plugin synchronize.
## How does this plug-in synchronise.
![Synchronization](../images/1.png)
![Synchronisation](../images/1.png)
1. When notes are created or modified, Obsidian raises some events. Self-hosted LiveSync catches these events and reflects changes into Local PouchDB.
2. PouchDB automatically or manually replicates changes to remote CouchDB.

View File

@@ -2,7 +2,7 @@
## 这个插件是怎么实现同步的.
![Synchronization](../images/1.png)
![Synchronisation](../images/1.png)
1. 当笔记创建或修改时Obsidian会触发事件。Self-hosted LiveSync捕获这些事件并将变更同步至本地PouchDB
2. PouchDB通过自动或手动方式将变更同步至远程CouchDB

View File

@@ -2,7 +2,7 @@
## 同期
![Synchronization](../images/1.png)
![Synchronisation](../images/1.png)
1. ートが更新された際、Obsidianがイベントを発報します。Obsidian-LiveSyncはそれをハンドリングして、ローカルのPouchDBに変更を反映します。
2. PouchDBは、リモートのCouchDBに差分をレプリケーションします。

View File

@@ -2,23 +2,102 @@
## Spelling and Vocabulary conventions
1. Almost all of the english words are written in British English. For example, "organisation" instead of "organization", "synchronisation" instead of "synchronization", etc. This convention originated from the author's personal preference but is now maintained for consistency.
All guidelines and conventions listed below are disclosed and maintained solely for the sake of documentation `consistency`.
2. Idiomatic terms, such as used in HTML, CSS, and JavaScript, are usually be aligned with the language used in the technology. For example, "color" instead of "colour", "program" instead of "programme", etc. Especially, terms which are used for attributes, properties, and methods are notable.
1. Almost all of the English words are written in British English. This convention originated from the author's personal preference.
- **Traditional Spelling (Trad-spelling)**: We prefer traditional British English spellings. In particular, we use `-ise` and `-isation` suffixes rather than the Oxford spelling `-ize` and `-ization` (for example, 'initialisation', 'synchronisation', and 'organisation').
- **Oxford Comma**: We use the serial (Oxford) comma to separate items in lists of three or more (for example, 'settings, snippets, and themes' instead of 'settings, snippets and themes').
- **Logical Punctuation**: We place punctuation marks (such as commas and full stops) outside quotation marks, unless the punctuation mark is part of the quoted text itself. For example, we write 'dialogue', not 'dialogue,'.
- **BBC News Styleguide**: If in wonder, the BBC News Styleguide may be useful as a reference.
2. Idiomatic terms, such as those used in HTML, CSS, and JavaScript, are usually aligned with the language used in the technology. For example, "color" instead of "colour", "program" instead of "programme", etc. Especially, terms which are used for attributes, properties, and methods are notable.
3. We use `dialogue` in documentation for consistency. While `dialog` may appear in source code, particularly in class names, method names, and attributes (following technical conventions in No. 2), we consistently use `dialogue` for user-facing messages and general documentation text. This approach balances No. 1 with No. 2.
4. Contractions are not used. For example, "do not" instead of "don't", "cannot" instead of "can't", etc. especially `'d`.
4. Contractions are not used. For example, "do not" instead of "don't", "cannot" instead of "can't", etc., especially `'d`.
- We may encounter difficulties with tenses.
5. However, try using affirmative forms, `Discard` instead of `Do not keep`, `Continue` instead of `Do not stop`, etc.
- Some languages, such as Japanese, have a different meaning for `yes` and `no` between affirmative and negative questions.
## Terminology
6. Single quotation marks (`'`) are preferred over double quotation marks (`"`) in general documentation text, unless the context requires double quotes (for example, inside JSON code blocks).
- Self-hosted LiveSync
- This plug-in name. `Self-hosted` is one word.
### Terminology
- Boot-up sequence (boot-sequence)
- The initialisation process of the plug-in when Obsidian starts. It starts with the loading of the plug-in, setting up core services, loading saved settings, and opening the local database. Once the layout is ready, the plug-in checks for the presence of flag files, runs configuration diagnostics, connects to the remote database, and begins file watching. The sequence finishes once the plug-in is fully ready and operational.
- Broken files (Size mismatch)
- A state where a file's metadata and the actual content stored in its chunks do not match, causing file retrieval or synchronisation failures. These mismatches can be detected and resolved by running validation tools such as `Verify and repair all files` on the Hatch pane.
- Chunk / Chunks
- Divided units of data stored in the database or object storage to facilitate efficient synchronisation.
- Compaction
- A database maintenance procedure that discards old historical document revisions to shrink the remote database size.
- Custom HTTP Handler / Use Internal API (CORS Bypass Settings)
- Settings used to bypass CORS restrictions by routing requests through Obsidian's native request APIs. There are two distinct settings under the hood depending on the remote server type:
- **For S3-compatible Object Storage (useCustomRequestHandler)**: Labeled as **"Use Custom HTTP Handler"** in the standard settings tab, **"Use internal API"** in the Svelte-based Setup Wizard dialogue, and represented as `useProxy` in the Setup URI's query parameters due to an unfortunate misunderstanding during development.
- **For CouchDB (useRequestAPI)**: Labeled as **"Use Request API to avoid `inevitable` CORS problem"** in the standard settings tab, **"Use Internal API"** in the Svelte-based Setup Wizard dialogue, and represented as `useRequestAPI` in the Setup URI's query parameters.
- Customisation Sync
- The feature that synchronises settings, snippets, themes, and plug-ins. Write with an "s" in documentation (`Customisation`), though technical configurations and links may use `customization`.
- Database Adapter (IDB vs. IndexedDB)
- The local database storage interface used by PouchDB. The `IDB` adapter is recommended since the older `IndexedDB` adapter is obsolete and known to cause memory leaks in `LiveSync` mode. Users can switch between these adapters without a full database rebuild, although a local data migration and an Obsidian restart are required.
- Database Suffix (additionalSuffixOfDatabaseName)
- A unique suffix appended to the database name to allow synchronising multiple vaults with the same name on the same remote server.
- E2EE Algorithm
- The cryptographic algorithm version used for end-to-end encryption. All devices in the synchronisation group must be configured with a compatible version (such as `V2` or `V1`).
- Eden (Eden Chunks)
- A performance optimisation where newly created chunks are held within the document until they stabilise, before graduating to independent chunks.
- Fast Setup (Simple Fetch)
- A simplified, automated initial synchronisation flow triggered when setting up subsequent devices or recovering a database. It bypasses the detailed step-by-step setup wizard dialogues, prompting the user with high-level data processing decisions and completing the initial download and local file scan in one continuous process.
- Flag files (redflag.md, redflag2.md, redflag3.md)
- Special Markdown files (or directories) placed at the root of the vault to stop the boot-up sequence or trigger recovery tasks. For instance, `redflag.md` suspends all processes, while `redflag2.md` (`flag_rebuild.md`) triggers a full database rebuild and `redflag3.md` (`flag_fetch.md`) discards the local database to fetch it again from the remote.
- Garbage Collection (GC)
- The process of identifying and purging unreferenced chunks (unused data) from local and remote databases to reclaim storage space.
- Hatch (Hatch pane)
- A dedicated troubleshooting and maintenance section in the plug-in settings, typically hidden behind a warning-labeled collapsible panel to prevent accidental misconfiguration. It contains diagnostic utilities, database reset controls, status reports, and advanced edge-case patches.
- Hidden File Sync
- The feature that synchronises files located in hidden directories (like `.obsidian`).
- JWT Authentication
- An experimental authentication option for CouchDB allowing secure token-based authentication instead of standard credentials. It requires a configured private key/secret, algorithm, expiration duration, subject, and key ID.
- LiveSync
- Very confusing term.
- As shorten-form of `Self-hosted LiveSync`.
- As a name of synchronisation mode. This should be changed to `Continuos`, in contrast to `Periodic`.
- A very confusing term.
- As a shortened form of `Self-hosted LiveSync`.
- As the name of a synchronisation mode. This should be changed to `Continuous`, in contrast to `Periodic`.
- livesync-serverpeer / webpeer
- Pseudo-clients that assist in WebRTC peer-to-peer communication.
- Metadata (File metadata)
- A database document that stores properties of a file, including its filename, path, size, modification time, conflict history, and references (hashes) of the chunks that comprise the file's content. In Self-hosted LiveSync, metadata is stored separately from the actual file content to enable efficient synchronisation and versioning.
- OneShot Sync
- A single, immediate bidirectional synchronisation (pull then push) triggered on demand or on specific events, as opposed to continuous (live) replication.
- Overwrite Server Data with This Device's Files
- A maintenance operation (formerly known as `Rebuild everything`) that discards the remote database and reconstructs it by uploading all current local files as a fresh database, overwriting any remote changes.
- Path Obfuscation
- A privacy option that encrypts file paths and folder names on the remote server.
- plug-in
- We use the hyphenated form `plug-in` in user-facing messages and general documentation, while `plugin` may appear in codebase files, configuration settings, or technical contexts.
- Relay Server (P2P relays)
- A WebSocket-based coordination server used to establish direct WebRTC peer-to-peer connections. The default relay is provided by the plug-in author.
- Remediation (maxMTimeForReflectEvents)
- A recovery setting that restricts the propagation of changes from the database to local storage, ignoring any file events (such as accidental mass deletions) that occurred after a specified date and time.
- Reset Synchronisation on This Device
- A maintenance operation (formerly known as `Fetch everything`) that discards the local database and reconstructs it by downloading all data from the remote server.
- Scram (Scram Switches)
- Emergency controls in the settings that allow users to suspend file watching or database writes to prevent corruption.
- Segmenter (Segmented-splitter)
- A chunking method that divides files on semantic boundaries (such as paragraphs or sections) rather than arbitrary byte boundaries.
- Self-hosted LiveSync
- The name of this plug-in. `Self-hosted` is one word.
- Setting Doctor (Config Doctor)
- A diagnostic utility that checks for mismatches or suboptimal configurations, presenting users with ideal values and recommendation reasons to easily resolve issues during migration, configuration import, or general troubleshooting.
- Setup URI
- An encrypted representation of the plug-in's settings containing server configuration, which allows users to clone their configuration across devices securely using a passphrase.
- Streaming replication (Stream-based replication)
- A data transfer method that downloads database documents as a continuous stream of events. It is significantly faster than traditional chunk-by-chunk HTTP requests and is used during Fast Setup to retrieve remote metadata quickly.
- Sync Mode
- The replication trigger mechanism. Users can select from `On Events` (synchronising on local file changes), `Periodic and Events` (synchronising at fixed intervals as well as on events), or `LiveSync` (continuous, real-time synchronisation).
- TURN Server (WebRTC P2P)
- A server type (Traversal Using Relays around NAT) used as a fallback to relay traffic when direct WebRTC peer-to-peer connection is blocked by strict NAT or firewalls.
- Update Thinning (Batch database update)
- An optimisation that groups multiple local file edits together over a short delay before committing them to the local database, reducing the number of database write operations.
- WebRTC P2P (Peer-to-Peer)
- A synchronisation method enabling direct communication between devices without a central server database.

65
docs/tips/fast-setup.md Normal file
View File

@@ -0,0 +1,65 @@
# Fast Setup (Simple Fetch)
Fast Setup is a streamlined, user-friendly data retrieval and initialisation flow designed to simplify setting up secondary devices or recovering databases.
Instead of guiding the user through the detailed multi-step setup wizard dialogues, Fast Setup prompts the user with high-level sync decisions and automates database download and local storage scanning in one continuous process.
---
## How It Works
When you import a **Setup URI** on a secondary device, or when a **Fetch All** operation is triggered (such as by placing a `redflag3.md` / `flag_fetch.md` flag file at the root of the vault), the plug-in schedules remote data retrieval.
On the next startup, the plug-in boots in scheduled fetch mode and opens a simplified dialogue: **"Data retrieval scheduled"**.
---
## Technical Characteristics
Fast Setup leverages several backend optimisations to make the retrieval fast, safe, and clean:
1. **Stream-based Replication for Speed**
- It fetches all remote metadata via stream reception, which is significantly faster than traditional chunk-by-chunk retrieval.
2. **Delayed File Reflection to Prevent Corrupted Warnings**
- By suspending file reflection during the download phase, it prevents the plug-in from raising temporary or false "corrupted data synchronisation" or "size mismatch" warnings that can occur during the chunk download process.
3. **Time-Based Comparison is Generally Sufficient**
- Since the vault is entering a fresh synchronisation or recovery state, comparing files based on their modification timestamps (newer-wins) is highly reliable and sufficient to reconcile files without needing complex manual conflict resolution.
---
## Step-by-Step Guide
### Step 1: Choose Data Processing Method
You will be prompted to choose how the retrieved remote data will interact with your existing local files:
1. **Compare time and take newer (newer-wins)**
- Compares the modified time of files and accepts the newer version.
- **Recommended if:** You have been using Self-hosted LiveSync and have made changes on multiple devices that you want to merge.
2. **Overwrite all with remote files (remote-wins)**
- Remote data is treated as the source of truth.
- **Recommended if:** You are setting up a brand new device with an empty or clean vault.
- *Warning: This will overwrite local files with remote files. Please ensure you have a backup of your local vault before proceeding.*
3. **Use the detailed flow (legacy)**
- Switches back to the detailed, traditional setup wizard dialogues.
- **Recommended if:** You want full control over the step-by-step database setup options.
### Step 2: Configure Conflict & Deletion Rules
Depending on your choice in Step 1, you will configure how to handle mismatches:
#### If you chose "Compare time and take newer":
- **Delete local files if they were deleted on remote**
- Keeps your local vault clean by removing files that have already been deleted on other devices.
- **Recreate remote files even if they were deleted on remote**
- Preserves local files and uploads them back to the remote database, even if they were deleted on other devices.
#### If you chose "Overwrite all with remote files":
- **Delete local files if not on remote**
- Removes local-only files so that your local vault matches the remote database exactly.
- **Keep local files even if not on remote**
- Retains all existing local-only files, although this may result in duplicates that you will need to clean up manually after synchronisation.
### Step 3: Automated Synchronisation
Once you confirm your choices:
1. The plug-in performs a fast download of the remote database (`fetchLocalDBFast`).
2. It automatically runs a full scan (`synchroniseAllFilesBetweenDBandStorage`) in the foreground to reflect database changes in your local vault files immediately.
3. The plug-in finalises the process and resumes normal operational status.

View File

@@ -0,0 +1,66 @@
# ファストセットアップ (Fast Setup / Simple Fetch)
ファストセットアップは、2台目以降のデバイスのセットアップやデータベース再構築時のデータ取得・初期化処理を、直感的かつ迅速に行うための簡略化された同期フローです。
従来のセットアップウィザードにおける複数の詳細なステップを踏むことなく、同期の基本方針を選択するだけで、データベースのダウンロードとローカルファイルのスキャン・反映を一連のプロセスとして自動的に実行します。
---
## 仕組み
2台目以降のデバイスで **セットアップURI** をインポートした場合や、手動でデータの再フェッチVaultルートに `redflag3.md` / `flag_fetch.md` を配置する等)が予約された場合、プラグインはリモートデータベースからのデータ取得スケジュールを設定します。
その後の起動時、プラグインはデータフェッチ予約モードで立ち上がり、**「Data retrieval scheduledデータ取得のスケジュール」** という簡略化されたダイアログを表示します。
---
## 技術的な特徴
ファストセットアップは、高速かつ安全でクリーンな処理を実現するために、以下の最適化を行っています。
1. **高速なストリーム受信**
- 全データを取得しますが、ストリーム受信によるレプリケーションを使用するため、処理が非常に高速です。
2. **ストレージ反映の遅延による不要な警告の抑制**
- データのダウンロード中にローカルファイル(ストレージ)への書き出し(反映)処理をあえて遅延させることで、同期途中で発生しがちな「破損データ同期」や「サイズ不一致」などの一時的なエラー警告を抑え込みます。
- すべてのデータダウンロードが完了した後に一括してストレージへ書き出すため、不必要な警告画面でユーザーを混乱させません。
3. **時刻ベース比較の妥当性**
- 初期セットアップやリカバリーの段階(この状態に移行した直後)においては、概ねファイル更新時刻(タイムスタンプ)ベースでの単純比較を行うことで、十分かつ妥当な同期結果を得ることができます。これにより複雑な競合解決の手間を省きます。
---
## 設定手順
### ステップ 1: データの反映方法の選択
取得したリモートデータを、既存のローカルファイルとどのように統合するかを選択します。
1. **Compare time and take newer (newer-wins)**
- ファイルの更新日時を比較し、より新しい方を採用します。
- **推奨されるケース:** すでに Self-hosted LiveSync を使用しており、複数のデバイスで編集した変更内容をタイムスタンプに基づいて統合したい場合。
2. **Overwrite all with remote files (remote-wins)**
- リモートデータベースの内容を正Source of Truthとして扱います。
- **推奨されるケース:** まったく新しいデバイスをセットアップする場合空のVaultなど
- *警告: ローカルにあるすべてのファイルがリモートの内容で上書きされます。重要なデータがある場合は、事前にバックアップを取得してください。*
3. **Use the detailed flow (legacy)**
- 従来の詳細なセットアップウィザードダイアログに戻ります。
- **推奨されるケース:** データベースの構成オプションをステップバイステップで細かく制御・確認したい場合。
### ステップ 2: 競合および削除ルールの構成
ステップ 1 での選択内容に応じて、ローカルとリモートの不一致をどう処理するかを設定します。
#### 「Compare time and take newer」を選択した場合:
- **Delete local files if they were deleted on remote**
- 他のデバイスで削除済みのファイルをローカルからも削除し、Vaultを同期・クリーンな状態に保ちます。
- **Recreate remote files even if they were deleted on remote**
- 他のデバイスで削除されたファイルであっても、ローカルファイルを維持し、リモートデータベースに再度アップロードします。
#### 「Overwrite all with remote files」を選択した場合:
- **Delete local files if not on remote**
- リモートに存在しないローカル専用ファイルを削除し、ローカルのVaultをリモートデータベースと完全に一致させます。
- **Keep local files even if not on remote**
- リモートに存在しないローカルファイルをそのまま残します。ただし、同期後に重複ファイルが発生する可能性があるため、その場合は手動でクリーンアップを行ってください。
### ステップ 3: 自動同期の実行
選択を確定すると、以下の処理が順次実行されます。
1. リモートデータベースの高速ダウンロードを実行します (`fetchLocalDBFast`)。
2. ローカルファイルへの変更反映のため、フォアグラウンドでフルスキャン (`synchroniseAllFilesBetweenDBandStorage`) を自動的に実行します。
3. 処理が完了すると、プラグインは通常の動作状態へ復帰します。

View File

@@ -224,7 +224,7 @@ There are many cases where this is really unclear. One possibility is that the c
- If you know when the files were deleted, set the time a bit before that.
- If not, bisecting may help us.
6. Delete `redflag.md`.
7. Perform `Reset synchronisation on This Device` on the `🎛️ Maintenance` pane.
7. Perform `Reset Synchronisation on This Device` on the `🎛️ Maintenance` pane.
This mode is very fragile. Please be careful.
@@ -236,16 +236,16 @@ not been stable. The new adapter has better performance and has a new feature
like purging. Therefore, we should use new adapters and current default is so.
However, when switching from an old adapter to a new adapter, some converting or
local database rebuilding is required, and it takes a few time. It was a long
local database rebuilding is required, and it takes some time. It was a long
time ago now, but we once inconvenienced everyone in a hurry when we changed the
format of our database. For these reasons, this toggle is automatically on if we
have upgraded from vault which using an old adapter.
When you rebuild everything or fetch from the remote again, you will be asked to
When you overwrite server data with this device's files or reset synchronisation on this device again, you will be asked to
switch this.
Therefore, experienced users (especially those stable enough not to have to
rebuild the database) may have this toggle enabled in their Vault. Please
overwrite server data) may have this toggle enabled in their Vault. Please
disable it when you have enough time.
### ZIP (or any extensions) files were not synchronised. Why?
@@ -255,14 +255,20 @@ It depends on Obsidian detects. May toggling `Detect all extensions` of
### I hope to report the issue, but you said you needs `Report`. How to make it?
We can copy the report to the clipboard, by pressing the `Make report` button on
the `Hatch` pane. ![Screenshot](../images/hatch.png)
We can copy the report to the clipboard, by performing
`Generate full report for opening the issue with debug info` command!
### Where can I check the log?
We can launch the log pane by `Show log` on the command palette. And if you have
troubled something, please enable the `Verbose Log` on the `General Setting`
pane.
`Generate full report for opening the issue with debug info` command also contains
the recent 1000 log lines, which is very helpful for debugging. Full-report is
already set to the verbose level, so it contains all the logs without enabling the
`Verbose Log` toggle.
Let me note that please be sure to remove any sensitive information before sharing the report.
However, the logs would not be kept so long and cleared when restarted. If you
want to check the logs, please enable `Write logs into the file` temporarily.
@@ -297,9 +303,9 @@ happened on other devices. This means that conflicts will happen in the past,
after the time we have synchronised. Hence we cannot collect and delete the
unused chunks even though if we are not currently referenced.
To shrink the database size, `Rebuild everything` only reliably and effectively.
To shrink the database size, `Overwrite Server Data with This Device's Files` is the only reliable and effective way.
But do not worry, if we have synchronised well. We have the actual and real
files. Only it takes a bit of time and traffics.
files. Only it takes a bit of time and traffic.
### How to launch the DevTools
@@ -367,47 +373,64 @@ without Obsidian.
For example, if there is `redflag.md`, Self-hosted LiveSync suspends all database and storage
processes.
### Flag Files
### Scram State and Flag Files (SCRAM Warning Loop)
The flag file is a simple Markdown file designed to prevent storage events and database events in self-hosted LiveSync.
Its very existence is significant; it may be left blank, or it may contain text; either is acceptable.
The plug-in uses a **Scram state** (emergency suspension of all synchronisation processes) to prevent database corruption when severe errors or conflicts are detected. This state is often triggered or persisted by **flag files** placed at the root of the vault.
This file is in Markdown format so that it can be placed in the Vault externally, even if Obsidian fails to launch.
If you encounter a warning saying **"Scram detected, all sync operations are suspended per SCRAM"** or get caught in an infinite loop where the warning persists even after clicking "Resume", it is likely due to a flag file in your vault.
There are some options to use `redflag.md`.
#### Flag Files
| Filename | Human-Friendly Name | Description |
| ------------- | ------------------- | ------------------------------------------------------------------------------------ |
| `redflag.md` | - | Suspends all processes. |
| `redflag2.md` | `flag_rebuild.md` | Suspends all processes, and rebuild both local and remote databases by local files. |
| `redflag3.md` | `flag_fetch.md` | Suspends all processes, discard the local database, and fetch from the remote again. |
A flag file is a simple Markdown file located at the root of your vault. Its very existence is significant; it may be left blank or contain any text. These files are used so they can be easily placed or deleted from outside Obsidian (e.g., when Obsidian fails to launch).
When fetching everything remotely or performing a rebuild, restarting Obsidian
is performed once for safety reasons. At that time, Self-hosted LiveSync uses
these files to determine whether the process should be carried out. (The use of
normal markdown files is a trick to externally force cancellation in the event
of faults in the rebuild or fetch function itself, especially on mobile
devices). This mechanism is also used for set-up. And just for information,
these files are also not subject to synchronisation.
| Filename | Human-Friendly Name | Description |
| ------------- | ------------------- | --------------------------------------------------------------------------------------- |
| `redflag.md` | - | Suspends all processes (activates Scram). |
| `redflag2.md` | `flag_rebuild.md` | Suspends all processes, and overwrites server data with this device's files. |
| `redflag3.md` | `flag_fetch.md` | Suspends all processes, discards the local database, and resets synchronisation on this device. |
However, occasionally the deletion of files may fail. This should generally work
normally after restarting Obsidian. (As far as I can observe).
When resetting synchronisation on this device or overwriting server data, restarting Obsidian is performed once for safety reasons. At that time, Self-hosted LiveSync uses these files to determine whether the process should be carried out. (This mechanism is especially useful on mobile devices to force cancellation if the database rebuilding fails). These files are not subject to synchronisation.
#### How to Resolve the Scram Loop
If you cannot disable Scram, please follow these steps:
1. Close Obsidian completely.
2. Open your system's file manager and check the root directory of your vault.
3. Locate and delete any flag files (such as `redflag.md`, `redflag2.md`, or `redflag3.md`).
4. Launch Obsidian.
5. Go to the settings dialogue -> **Hatch** -> **Scram Switches**, and manually toggle **Suspend file watching** and **Suspend database reflecting** to `false` (disabled) if they have not been reset automatically.
> [!TIP]
> This is the reason why flag files are standard `.md` files: it allows you to manage them externally. On mobile devices, you can use system file manager applications (such as the native **Files** app on iOS/iPadOS or **Files by Google** on Android) to find and delete these files to resolve a lock, or conversely, create/place a new `redflag.md` file (or directory) at the root of your vault to force-suspend synchronisation and stop Obsidian's boot-up sequence if you need to fix a database issue.
### JWT Authentication Errors
#### DataError when configuring JWT authentication
If you encounter a `DataError:` with no additional information in the logs when configuring JWT authentication, this usually indicates a private key formatting issue.
Self-hosted LiveSync requires the private key (for ES256/ES512 algorithms) to be in the **PKCS#8 PEM** format. Standard SEC1 EC private keys (which begin with `-----BEGIN EC PRIVATE KEY-----`) will trigger this error.
To resolve this, convert your private key to PKCS#8 format using the following `openssl` command:
```bash
openssl pkcs8 -topk8 -inform PEM -nocrypt -in private.key -out pkcs8.key
```
Then paste the contents of `pkcs8.key` (which begins with `-----BEGIN PRIVATE KEY-----`) into the JWT Key field.
### Old tips
- Rarely, a file in the database could be corrupted. The plugin will not write
- Rarely, a file in the database could be corrupted. The plug-in will not write
to local storage when a file looks corrupted. If a local version of the file
is on your device, the corruption could be fixed by editing the local file and
synchronizing it. But if the file does not exist on any of your devices, then
synchronising it. But if the file does not exist on any of your devices, then
it can not be rescued. In this case, you can delete these items from the
settings dialog.
settings dialogue.
- To stop the boot-up sequence (eg. for fixing problems on databases), you can
put a `redflag.md` file (or directory) at the root of your vault. Tip for iOS:
a redflag directory can be created at the root of the vault using the File
application.
- Also, with `redflag2.md` placed, we can automatically rebuild both the local
and the remote databases during the boot-up sequence. With `redflag3.md`, we
can discard only the local database and fetch from the remote again.
- Also, with `redflag2.md` placed, we can automatically overwrite server data with this device's files during the boot-up sequence. With `redflag3.md`, we
can discard only the local database and reset synchronisation on this device.
- Q: The database is growing, how can I shrink it down? A: each of the docs is
saved with their past 100 revisions for detecting and resolving conflicts.
Picturing that one device has been offline for a while, and comes online
@@ -419,7 +442,7 @@ normally after restarting Obsidian. (As far as I can observe).
So, We have to make the database again like an enlarged git repo if you want
to solve the root of the problem.
- And more technical Information is in the [Technical Information](tech_info.md)
- If you want to synchronize files without obsidian, you can use
- If you want to synchronise files without obsidian, you can use
[filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync).
- WebClipper is also available on Chrome Web
Store:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)

View File

@@ -2,7 +2,6 @@
import esbuild from "esbuild";
import process from "process";
import builtins from "builtin-modules";
import sveltePlugin from "esbuild-svelte";
import { sveltePreprocess } from "svelte-preprocess";
import fs from "node:fs";

View File

@@ -1,102 +1,134 @@
import typescriptEslint from "@typescript-eslint/eslint-plugin";
import svelte from "eslint-plugin-svelte";
import _import from "eslint-plugin-import";
import { fixupPluginRules } from "@eslint/compat";
import tsParser from "@typescript-eslint/parser";
import path from "node:path";
import { fileURLToPath } from "node:url";
import js from "@eslint/js";
import { FlatCompat } from "@eslint/eslintrc";
import obsidianmd from "eslint-plugin-obsidianmd";
import globals from "globals";
import { defineConfig, globalIgnores } from "eslint/config";
import * as sveltePlugin from "eslint-plugin-svelte";
import svelteParser from "svelte-eslint-parser";
const warnWhileDev = "off"; // Change to "warn" to enable warnings for rules that are currently disabled.
export default defineConfig([
globalIgnores([
// Build outputs and legacy files
"**/build",
"coverage",
"**/main.js",
"main_org.js",
"pouchdb-browser.js",
"version-bump.mjs",
"package.json",
"**/*.json",
"**/.eslintrc.js.bak",
// Files from linked dependencies (those files should not exist for most people).
"modules/octagonal-wheels/dist/**/*",
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const compat = new FlatCompat({
baseDirectory: __dirname,
recommendedConfig: js.configs.recommended,
allConfig: js.configs.all,
});
// Sub-projects (Exclude from root linting as they have different environments)
"src/apps/**/*",
"utils/**/*",
export default [
// Specific exclusions from common library (src/lib)
"src/lib/coverage",
"src/lib/browsertest",
"src/lib/test",
"src/lib/_tools",
"src/lib/src/patches/pouchdb-utils",
"src/lib/src/cli",
"src/lib/src/services/implements/browser/**",
"src/lib/src/services/implements/headless/**",
"src/lib/src/API",
// Config files and build scripts
"**/jest.config.js",
"**/rollup.config.js",
"**/esbuild.config.mjs",
"**/terser.*.mjs",
".prettierrc.*.mjs",
".prettierrc.mjs",
"*.config.mjs",
"vite.*",
"vitest.*",
// Testing files (Simplified patterns)
"test/**",
"**/*.test.ts",
"**/*.unit.spec.ts",
"**/test.ts",
"**/tests.ts",
]),
...sveltePlugin.configs["flat/base"],
...obsidianmd.configs.recommended,
{
ignores: [
"**/node_modules/*",
"**/jest.config.js",
"src/lib/coverage",
"src/lib/browsertest",
"**/test.ts",
"**/tests.ts",
"**/**test.ts",
"**/**.test.ts",
"**/esbuild.*.mjs",
"**/terser.*.mjs",
"**/node_modules",
"**/build",
"**/.eslintrc.js.bak",
"src/lib/src/patches/pouchdb-utils",
"**/esbuild.config.mjs",
"**/rollup.config.js",
"modules/octagonal-wheels/rollup.config.js",
"modules/octagonal-wheels/dist/**/*",
"src/lib/test",
"src/lib/src/cli",
"**/main.js",
"src/apps/**/*",
".prettierrc.*.mjs",
".prettierrc.mjs",
"*.config.mjs"
],
},
...compat.extends(
"eslint:recommended",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:@typescript-eslint/recommended"
),
{
plugins: {
"@typescript-eslint": typescriptEslint,
svelte,
import: fixupPluginRules(_import),
},
files: ["**/*.ts"],
// ignores:["src/lib/**/*.ts"], // Exclude library files from root linting (they have different environments and rules).
languageOptions: {
globals: { ...globals.browser, PouchDB: "readonly" },
parser: tsParser,
ecmaVersion: 5,
sourceType: "module",
parserOptions: {
project: ["tsconfig.json"],
project: "./tsconfig.json",
},
},
linterOptions:{
reportUnusedDisableDirectives: false,
},
rules: {
// -- Base rules (turned off in favour of TS specific versions or explicitly disabled).
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": [
"error",
{
args: "none",
},
],
"no-unused-labels": "off",
"@typescript-eslint/ban-ts-comment": "off",
"no-prototype-builtins": "off",
"@typescript-eslint/no-empty-function": "off",
"require-await": "error",
"@typescript-eslint/require-await": "warn",
"@typescript-eslint/no-misused-promises": "warn",
"@typescript-eslint/no-floating-promises": "warn",
"no-async-promise-executor": "warn",
"require-await": "off",
// -- TypeScript specific rules
// @typescript-eslint/no-unsafe-* rules and @typescript-eslint/no-explicit-any:
// This project contains a lot of library-sh code where the use of `any` is often necessary and justified.
// Rules is now set to 'off' for a while.
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/no-unsafe-argument": "off",
"@typescript-eslint/no-unsafe-call": "off",
"@typescript-eslint/no-unsafe-member-access": "off",
"@typescript-eslint/no-unsafe-return": "off",
"@typescript-eslint/no-unsafe-assignment": "off",
// -- Reasonable rules.
"@typescript-eslint/no-deprecated": warnWhileDev,
"@typescript-eslint/no-unused-vars": ["error", { args: "none" }],
"@typescript-eslint/ban-ts-comment": "off",
"@typescript-eslint/no-empty-function": "off",
"@typescript-eslint/require-await": "error",
"@typescript-eslint/no-misused-promises": "error",
"@typescript-eslint/no-floating-promises": "error",
"@typescript-eslint/no-unnecessary-type-assertion": "error",
"no-constant-condition": [
"error",
{
checkLoops: false,
},
],
// -- Obsidian rules
// obsidianmd/no-unsupported-api: usually this project checks for API support at runtime, so this rule is not critical but can be helpful to catch potential issues.
"obsidianmd/no-unsupported-api": warnWhileDev,
// -- General rules
"no-async-promise-executor": warnWhileDev,
"no-constant-condition": ["error", { checkLoops: false }],
// -- Disabled rules
// no-undef: This option breaks the global declarations for the library files and is not worth the effort to fix at this time.
"no-undef": "off",
// -- Plugin specific overrides
"obsidianmd/rule-custom-message": "off",
"obsidianmd/ui/sentence-case": "off",
"obsidianmd/no-plugin-as-component": "off",
// -- Temporary overrides for migration
"obsidianmd/no-static-styles-assignment": "off",
},
},
];
{
files: ["**/*.svelte"],
languageOptions: {
parser: svelteParser,
parserOptions: {
parser: tsParser,
extraFileExtensions: [".svelte"],
},
},
rules: {
// no-unused-vars:
// Svelte template's declarations have a lot of false positives and the rule is not worth the effort to fix at this time.
// it may improve in the future with some options as like ["error", { argsIgnorePattern: "^_", varsIgnorePattern: "^_" }],]
"no-unused-vars": "off",
"obsidianmd/no-plugin-as-component": "off",
"obsidianmd/ui/sentence-case": "off",
},
},
]);

View File

@@ -1,8 +1,8 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.25.58",
"minAppVersion": "0.9.12",
"version": "0.25.73",
"minAppVersion": "1.7.2",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",
"authorUrl": "https://github.com/vrtmrz",

1851
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.25.58",
"version": "0.25.73",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -19,13 +19,13 @@
"buildVite": "npx dotenv-cli -e .env -- vite build --mode production",
"buildViteOriginal": "npx dotenv-cli -e .env -- vite build --mode original",
"buildDev": "node esbuild.config.mjs dev",
"lint": "eslint src",
"lint": "eslint --cache --concurrency auto src",
"svelte-check": "svelte-check --tsconfig ./tsconfig.json",
"tsc-check": "tsc --noEmit",
"pretty": "npm run prettyNoWrite -- --write --log-level error",
"prettyCheck": "npm run prettyNoWrite -- --check",
"prettyNoWrite": "prettier --config ./.prettierrc.mjs \"**/*.js\" \"**/*.ts\" \"**/*.json\" ",
"check": "npm run lint && npm run svelte-check",
"check": "npm run tsc-check && npm run lint && npm run svelte-check",
"unittest": "deno test -A --no-check --coverage=cov_profile --v8-flags=--expose-gc --trace-leaks ./src/",
"test": "vitest run",
"test:unit": "vitest run --config vitest.config.unit.ts",
@@ -54,21 +54,21 @@
"test:docker-all:start": "npm run test:docker-all:up && sleep 5 && npm run test:docker-all:init",
"test:docker-all:stop": "npm run test:docker-all:down",
"test:full": "npm run test:docker-all:start && vitest run --coverage && npm run test:docker-all:stop",
"test:p2p": "bash test/suitep2p/run-p2p-tests.sh"
"test:p2p": "bash test/suitep2p/run-p2p-tests.sh",
"version": "node version-bump.mjs && git add manifest.json versions.json"
},
"keywords": [],
"author": "vorotamoroz",
"license": "MIT",
"devDependencies": {
"@chialab/esbuild-plugin-worker": "^0.19.0",
"@eslint/compat": "^2.0.2",
"@eslint/eslintrc": "^3.3.4",
"@eslint/js": "^9.39.3",
"@sveltejs/vite-plugin-svelte": "^6.2.4",
"@tsconfig/svelte": "^5.0.8",
"@types/deno": "^2.5.0",
"@types/diff-match-patch": "^1.0.36",
"@types/markdown-it": "^14.1.2",
"@types/micromatch": "^4.0.10",
"@types/node": "^24.10.13",
"@types/pouchdb": "^6.4.2",
"@types/pouchdb-adapter-http": "^6.1.6",
@@ -80,21 +80,18 @@
"@types/transform-pouch": "^1.0.6",
"@typescript-eslint/eslint-plugin": "8.56.1",
"@typescript-eslint/parser": "8.56.1",
"@vitest/browser": "^4.1.1",
"@vitest/browser-playwright": "^4.1.1",
"@vitest/coverage-v8": "^4.1.1",
"builtin-modules": "5.0.0",
"dotenv": "^17.3.1",
"@vitest/browser": "^4.1.8",
"@vitest/browser-playwright": "^4.1.8",
"@vitest/coverage-v8": "^4.1.8",
"dotenv-cli": "^11.0.0",
"esbuild": "0.25.0",
"esbuild-plugin-inline-worker": "^0.1.1",
"esbuild-svelte": "^0.9.4",
"eslint": "^9.39.3",
"eslint-plugin-import": "^2.32.0",
"eslint-plugin-obsidianmd": "^0.3.0",
"eslint-plugin-svelte": "^3.15.0",
"events": "^3.3.0",
"glob": "^13.0.6",
"obsidian": "^1.12.3",
"globals": "^14.0.0",
"playwright": "^1.58.2",
"postcss": "^8.5.6",
"postcss-load-config": "^6.0.1",
@@ -115,13 +112,14 @@
"svelte-check": "^4.4.3",
"svelte-preprocess": "^6.0.3",
"terser": "^5.39.0",
"tinyglobby": "^0.2.15",
"transform-pouch": "^2.0.0",
"tslib": "^2.8.1",
"tsx": "^4.21.0",
"typescript": "5.9.3",
"vite": "^7.3.1",
"vite-plugin-istanbul": "^8.0.0",
"vitest": "^4.1.1",
"vitest": "^4.1.8",
"webdriverio": "^9.27.0",
"yaml": "^2.8.2"
},
@@ -132,17 +130,21 @@
"@smithy/middleware-apply-body-checksum": "^4.3.9",
"@smithy/protocol-http": "^5.3.9",
"@smithy/querystring-builder": "^4.2.9",
"@trystero-p2p/nostr": "^0.23.0",
"@smithy/util-retry": "^4.4.5",
"@trystero-p2p/nostr": "^0.24.0",
"chokidar": "^4.0.0",
"commander": "^14.0.3",
"diff-match-patch": "^1.0.5",
"fflate": "^0.8.2",
"idb": "^8.0.3",
"markdown-it": "^14.1.1",
"micromatch": "^4.0.0",
"minimatch": "^10.2.2",
"octagonal-wheels": "^0.1.45",
"obsidian": "^1.12.3",
"octagonal-wheels": "^0.1.46",
"pouchdb-adapter-leveldb": "^9.0.0",
"qrcode-generator": "^1.4.4",
"werift": "^0.22.9",
"werift": "^0.23.0",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
}
}

View File

@@ -1,4 +1,5 @@
import { LOG_LEVEL_INFO } from "octagonal-wheels/common/logger";
import type PouchDB from "pouchdb-core";
import type { SimpleStore } from "octagonal-wheels/databases/SimpleStoreBase";
import type { HasSettings, ObsidianLiveSyncSettings, EntryDoc } from "./lib/src/common/types";
import { __$checkInstanceBinding } from "./lib/src/dev/checks";
@@ -123,7 +124,7 @@ export class LiveSyncBaseCore<
for (const module of this.modules) {
if (module.constructor === constructor) return module as T;
}
throw new Error(`Module ${constructor} not found or not loaded.`);
throw new Error(`Module ${constructor.name} not found or not loaded.`);
}
/**
@@ -160,8 +161,10 @@ export class LiveSyncBaseCore<
module.onBindFunction(this, this.services);
__$checkInstanceBinding(module); // Check if all functions are properly bound, and log warnings if not.
} else {
// module should not be never.
const moduleName = (module as unknown)?.constructor?.name ?? "unknown";
this.services.API.addLog(
`Module ${(module as any)?.constructor?.name ?? "unknown"} does not have onBindFunction, skipping binding.`,
`Module ${moduleName} does not have onBindFunction, skipping binding.`,
LOG_LEVEL_INFO
);
}

View File

@@ -3,4 +3,6 @@ test/*
!test/*.sh
test/test-init.local.sh
node_modules
.*.json
.*.json
*.env
!.test.env

View File

@@ -60,7 +60,7 @@ RUN apt-get update \
WORKDIR /build
# Install workspace dependencies first (layer-cache friendly)
COPY package.json ./
COPY package.json package-lock.json ./
RUN npm install
# Copy the full source tree and build the CLI bundle

View File

@@ -45,11 +45,90 @@ CLI Main
- Settings management (JSON file)
- Graceful shutdown handling
## Something I realised later that could lead to misunderstandings
## Usage
The term `vault` in this README refers to the directory containing your local database and settings file. Not the actual files you want to sync. I will fix this later, but please be mind this for now.
The CLI operates on a **database directory** which contains PouchDB data and settings.
## Docker
> [!NOTE]
> `livesync-cli` is the alias for the CLI executable. Please replace with the actual command of your installation (e.g. `npm run --silent cli --` or `docker run ...`).
```bash
livesync-cli [database-path] [command] [args...]
```
### Arguments
- `database-path`: Path to the directory where `.livesync` folder and `settings.json` are (or will be) located.
- Note: In previous versions, this was referred to as the "vault" path. Now it is clearly distinguished from the actual vault (the directory containing your `.md` files).
### Commands
- `sync`: Run one replication cycle with the remote CouchDB.
- `mirror [vault-path]`: Bidirectional sync between the local database and a local directory (**the actual vault**).
- If `vault-path` is provided, the CLI will synchronise the database with files in the vault directory.
- If `vault-path` is omitted, it defaults to `database-path` (compatibility mode).
- Use this command to keep your local `.md` files in sync with the database.
- `ls [prefix]`: List files currently stored in the local database.
- `push <src> <dst>`: Push a local file `<src>` into the database at path `<dst>`.
- `pull <src> <dst>`: Pull a file `<src>` from the database into local file `<dst>`.
- `cat <src>`: Read a file from the database and write to stdout.
- `put <dst>`: Read from stdin and write to the database path `<dst>`.
- `remote-add <name> <connstr>`: Add a remote configuration from a connection string.
- `remote-rm <remote-id>`: Remove a remote configuration by ID.
- `remote-ls`: List remote configurations (`id`, `name`, `active/inactive`, redacted URI).
- `remote-export <remote-id>`: Export the stored connection string by remote ID.
- `remote-set <remote-id> <connstr>`: Replace the stored connection string by remote ID.
- `remote-activate <remote-id>`: Activate a remote configuration by ID.
- `init-settings [file]`: Create a default settings file.
### Examples
```bash
# Basic sync with remote
livesync-cli ./my-db sync
# Mirroring to your actual Obsidian vault
livesync-cli ./my-db mirror /path/to/obsidian-vault
# Manual file operations
livesync-cli ./my-db push ./note.md folder/note.md
livesync-cli ./my-db pull folder/note.md ./note.md
```
## Installation
### Build from source
```bash
# Clone with submodules, because the shared core lives in src/lib
git clone --recurse-submodules <repository-url>
cd obsidian-livesync
# If you already cloned without submodules, run this once instead
git submodule update --init --recursive
# Install dependencies from the repository root
npm install
# Build the CLI from its package directory
cd src/apps/cli
npm run build
```
If `src/lib` is missing, `npm run build` now stops early with a targeted message
instead of a low-level Vite `ENOENT` error.
Run the CLI:
```bash
# Run with npm script (from repository root)
npm run --silent cli -- [database-path] [command] [args...]
# Run the built executable directly
node src/apps/cli/dist/index.cjs [database-path] [command] [args...]
```
### Docker
A Docker image is provided for headless / server deployments. Build from the repository root:
@@ -61,28 +140,28 @@ Run:
```bash
# Sync with CouchDB
docker run --rm -v /path/to/your/vault:/data livesync-cli sync
docker run --rm -v /path/to/your/db:/data livesync-cli sync
# Mirror to a specific vault directory
docker run --rm -v /path/to/your/db:/data -v /path/to/your/vault:/vault livesync-cli mirror /vault
# List files in the local database
docker run --rm -v /path/to/your/vault:/data livesync-cli ls
# Generate a default settings file
docker run --rm -v /path/to/your/vault:/data livesync-cli init-settings
docker run --rm -v /path/to/your/db:/data livesync-cli ls
```
The vault directory is mounted at `/data` by default. Override with `-e LIVESYNC_DB_PATH=/other/path`.
The database directory is mounted at `/data` by default. Override with `-e LIVESYNC_DB_PATH=/other/path`.
### P2P (WebRTC) and Docker networking
#### P2P (WebRTC) and Docker networking
The P2P replicator (`p2p-host`, `p2p-sync`, `p2p-peers`) uses WebRTC and generates
three kinds of ICE candidates. The default Docker bridge network affects which
candidates are usable:
| Candidate type | Description | Bridge network |
|---|---|---|
| `host` | Container bridge IP (`172.17.x.x`) | Unreachable from LAN peers |
| `srflx` | Host public IP via STUN reflection | Works over the internet |
| `relay` | Traffic relayed via TURN server | Always reachable |
| Candidate type | Description | Bridge network |
| -------------- | ---------------------------------- | -------------------------- |
| `host` | Container bridge IP (`172.17.x.x`) | Unreachable from LAN peers |
| `srflx` | Host public IP via STUN reflection | Works over the internet |
| `relay` | Traffic relayed via TURN server | Always reachable |
**LAN P2P on Linux** — use `--network host` so that the real host IP is
advertised as the `host` candidate:
@@ -91,6 +170,8 @@ advertised as the `host` candidate:
docker run --rm --network host -v /path/to/your/vault:/data livesync-cli p2p-host
```
Note: also fix the alias to include `--network host` if you want to use `livesync-cli` for P2P commands.
> `--network host` is not available on Docker Desktop for macOS or Windows.
**LAN P2P on macOS / Windows Docker Desktop** — configure a TURN server in the
@@ -103,16 +184,35 @@ candidate carries the host's public IP and peers can connect normally.
**CouchDB sync only (no P2P)** — no special network configuration is required.
## Installation
### Adding `livesync-cli` alias
To use the `livesync-cli` command globally, you can add an alias to your shell configuration file (e.g., `.zshrc` or `.bashrc`).
If you are using `npm run`, add the following line:
```bash
# Install dependencies (ensure you are in repository root directory, not src/apps/cli)
# due to shared dependencies with webapp and main library
npm install
# Build the project (ensure you are in `src/apps/cli` directory)
npm run build
alias livesync-cli='npm run --silent --prefix /path/to/repository/src/apps/cli cli --'
# or
alias livesync-cli="npm run --silent --prefix $PWD cli --"
```
Alternatively, if you want to use the built executable directly:
```bash
alias livesync-cli='node /path/to/repository/src/apps/cli/dist/index.cjs'
or
alias livesync-cli="node $PWD/dist/index.cjs"
```
If you prefer using Docker:
```bash
alias livesync-cli='docker run --rm -v /path/to/your/db:/data livesync-cli'
```
After adding the alias, restart your shell or run `source ~/.zshrc` (or `.bashrc`).
## Usage
### Basic Usage
@@ -121,43 +221,51 @@ As you know, the CLI is designed to be used in a headless environment. Hence all
```bash
# Sync local database with CouchDB (no files will be changed).
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json sync
livesync-cli /path/to/your-local-database --settings /path/to/settings.json sync
# Push files to local database
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json push /your/storage/file.md /vault/path/file.md
livesync-cli /path/to/your-local-database --settings /path/to/settings.json push /your/storage/file.md /vault/path/file.md
# Pull files from local database
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json pull /vault/path/file.md /your/storage/file.md
livesync-cli /path/to/your-local-database --settings /path/to/settings.json pull /vault/path/file.md /your/storage/file.md
# Verbose logging
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json --verbose
livesync-cli /path/to/your-local-database --settings /path/to/settings.json --verbose
# Apply setup URI to settings file (settings only; does not run synchronisation)
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json setup "obsidian://setuplivesync?settings=..."
livesync-cli /path/to/your-local-database --settings /path/to/settings.json setup "obsidian://setuplivesync?settings=..."
# Put text from stdin into local database
echo "Hello from stdin" | npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json put /vault/path/file.md
echo "Hello from stdin" | livesync-cli /path/to/your-local-database --settings /path/to/settings.json put /vault/path/file.md
# Output a file from local database to stdout
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json cat /vault/path/file.md
livesync-cli /path/to/your-local-database --settings /path/to/settings.json cat /vault/path/file.md
# Output a specific revision of a file from local database
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json cat-rev /vault/path/file.md 3-abcdef
livesync-cli /path/to/your-local-database --settings /path/to/settings.json cat-rev /vault/path/file.md 3-abcdef
# Pull a specific revision of a file from local database to local storage
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json pull-rev /vault/path/file.md /your/storage/file.old.md 3-abcdef
livesync-cli /path/to/your-local-database --settings /path/to/settings.json pull-rev /vault/path/file.md /your/storage/file.old.md 3-abcdef
# List files in local database
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json ls /vault/path/
livesync-cli /path/to/your-local-database --settings /path/to/settings.json ls /vault/path/
# Show metadata for a file in local database
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json info /vault/path/file.md
livesync-cli /path/to/your-local-database --settings /path/to/settings.json info /vault/path/file.md
# Mark a file as deleted in local database
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json rm /vault/path/file.md
livesync-cli /path/to/your-local-database --settings /path/to/settings.json rm /vault/path/file.md
# Resolve conflict by keeping a specific revision
npm run --silent cli -- /path/to/your-local-database --settings /path/to/settings.json resolve /vault/path/file.md 3-abcdef
livesync-cli /path/to/your-local-database --settings /path/to/settings.json resolve /vault/path/file.md 3-abcdef
# Add/list/activate/remove remote configurations
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-add main "sls+https://user:pass@example.com/db"
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-ls
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-export remote-abc123
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-set remote-abc123 "sls+p2p://room-abc?passphrase=secret"
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-activate remote-abc123
livesync-cli /path/to/your-local-database --settings /path/to/settings.json remote-rm remote-abc123
```
### Configuration
@@ -192,7 +300,8 @@ The CLI uses the same settings format as the Obsidian plugin. Create a `.livesyn
```
Usage:
livesync-cli [database-path] [options] [command] [command-args]
livesync-cli <database-path> [options] <command> [command-args]
livesync-cli init-settings [path]
Arguments:
database-path Path to the local database directory (required except for init-settings)
@@ -201,9 +310,12 @@ Options:
--settings, -s <path> Path to settings file (default: .livesync/settings.json in local database directory)
--force, -f Overwrite existing file on init-settings
--verbose, -v Enable verbose logging
--debug, -d Enable debug logging (includes verbose)
--interval <N>, -i <N> (daemon only) Poll CouchDB every N seconds instead of using the _changes feed
--help, -h Show this help message
Commands:
daemon (default) Run mirror scan then continuously sync CouchDB <-> local filesystem
init-settings [path] Create settings JSON from DEFAULT_SETTINGS
sync Run one replication cycle and exit
p2p-peers <timeout> Show discovered peers as [peer]<TAB><peer-id><TAB><peer-name>
@@ -211,16 +323,16 @@ Commands:
p2p-host Start P2P host mode and wait until interrupted (Ctrl+C)
push <src> <dst> Push local file <src> into local database path <dst>
pull <src> <dst> Pull file <src> from local database into local file <dst>
pull-rev <src> <dst> <revision> Pull specific revision into local file <dst>
pull-rev <src> <dst> <rev> Pull specific revision <rev> into local file <dst>
setup <setupURI> Apply setup URI to settings file
put <vaultPath> Read text from standard input and write to local database
cat <vaultPath> Write latest file content from local database to standard output
cat-rev <vaultPath> <revision> Write specific revision content from local database to standard output
put <dst> Read text from standard input and write to local database path <dst>
cat <src> Write latest file content from local database to standard output
cat-rev <src> <rev> Write specific revision <rev> content from local database to standard output
ls [prefix] List files as path<TAB>size<TAB>mtime<TAB>revision[*]
info <vaultPath> Show file metadata including current and past revisions, conflicts, and chunk list
rm <vaultPath> Mark file as deleted in local database
resolve <vaultPath> <revision> Resolve conflict by keeping the specified revision
mirror <storagePath> <vaultPath> Mirror local file into local database.
info <path> Show file metadata including current and past revisions, conflicts, and chunk list
rm <path> Mark file as deleted in local database
resolve <path> <rev> Resolve conflict by keeping the specified revision
mirror [vaultPath] Mirror database contents to the local file system (vaultPath defaults to database-path)
```
Run via npm script:
@@ -300,16 +412,96 @@ In other words, it performs the following actions:
5. **Categorisation and synchronisation** — The union of both file sets is split into three groups and processed concurrently (up to 10 files at a time):
| Group | Condition | Action |
|---|---|---|
| **UPDATE DATABASE** | File exists in storage only | Store the file into the local database. |
| **UPDATE STORAGE** | File exists in database only | If the entry is active (not deleted) and not conflicted, restore the file from the database to storage. Deleted entries and conflicted entries are skipped. |
| **SYNC DATABASE AND STORAGE** | File exists in both | Compare `mtime` freshness. If storage is newer → write to database (`STORAGE → DB`). If database is newer → restore to storage (`STORAGE ← DB`). If equal → do nothing. Conflicted documents and files exceeding the size limit are always skipped. |
| Group | Condition | Action |
| ----------------------------- | ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **UPDATE DATABASE** | File exists in storage only | Store the file into the local database. |
| **UPDATE STORAGE** | File exists in database only | If the entry is active (not deleted) and not conflicted, restore the file from the database to storage. Deleted entries and conflicted entries are skipped. |
| **SYNC DATABASE AND STORAGE** | File exists in both | Compare `mtime` freshness. If storage is newer → write to database (`STORAGE → DB`). If database is newer → restore to storage (`STORAGE ← DB`). If equal → do nothing. Conflicted documents and files exceeding the size limit are always skipped. |
6. **Initialisation flag** — On the very first successful run, writes `initialized = true` to the key-value database so that subsequent runs can restore state in step 2.
Note: `mirror` does not respect file deletions. If a file is deleted in storage, it will be restored on the next `mirror` run. To delete a file, use the `rm` command instead. This is a little inconvenient, but it is intentional behaviour (if we handle this automatically in `mirror`, we should be against a ton of edge cases).
##### daemon
`daemon` is the default command when no command is specified. It runs an initial mirror scan and then continuously syncs changes in both directions:
- **CouchDB → local filesystem**: via the `_changes` feed (LiveSync mode, default) or periodic polling (`--interval N`).
- **local filesystem → CouchDB**: via chokidar file watching. Any file created, modified, or deleted in the vault directory is pushed to CouchDB.
In **LiveSync mode** the `_changes` feed delivers remote changes as they arrive, with sub-second latency. In **polling mode** (`--interval N`) the CLI polls CouchDB every N seconds. Use polling mode if your CouchDB instance does not support long-lived HTTP connections, or if you need predictable network usage.
The daemon exits cleanly on `SIGINT` or `SIGTERM`.
```bash
# LiveSync mode (default — _changes feed, near-real-time)
livesync-cli /path/to/vault
# Polling mode — poll every 60 seconds
livesync-cli /path/to/vault --interval 60
```
### .livesync/ignore
Place a `.livesync/ignore` file in your vault root to exclude files from sync in both directions (local → CouchDB and CouchDB → local).
**Format:**
- Lines beginning with `#` are comments.
- Blank lines are ignored.
- All other lines are [minimatch](https://github.com/isaacs/minimatch) glob patterns, relative to the vault root.
- The directive `import: .gitignore` (exactly this string) reads `.gitignore` from the vault root and merges its non-comment, non-blank lines into the ignore rules.
- Negation patterns (lines starting with `!`) are not supported and will cause an error on load.
**Example `.livesync/ignore`:**
```
# Ignore temporary files
*.tmp
*.swp
# Ignore build output
build/
dist/
# Merge patterns from .gitignore
import: .gitignore
```
Patterns apply in both directions: the chokidar watcher will not emit events for matched files, and the `isTargetFile` filter will exclude them from CouchDB → local sync.
Changes to this file require a daemon restart to take effect.
### Systemd Installation
The `deploy/` directory contains a systemd unit template and an install script.
**Automated install (user service, recommended):**
```bash
bash src/apps/cli/deploy/install.sh --vault /path/to/vault
```
**With polling interval:**
```bash
bash src/apps/cli/deploy/install.sh --vault /path/to/vault --interval 60
```
**System-wide install** (requires root / sudo for `/etc/systemd/system/`):
```bash
bash src/apps/cli/deploy/install.sh --system --vault /path/to/vault
```
The script:
1. Builds the CLI (`npm install` + `npm run build`).
2. Installs the binary to `~/.local/bin/livesync-cli` (user) or `/usr/local/bin/livesync-cli` (system).
3. Writes the unit file to `~/.config/systemd/user/livesync-cli.service` (user) or `/etc/systemd/system/livesync-cli.service` (system).
4. Runs `systemctl [--user] daemon-reload && systemctl [--user] enable --now livesync-cli`.
**Manual setup** — if you prefer to manage the unit yourself, copy `deploy/livesync-cli.service`, replace `LIVESYNC_BIN` and `LIVESYNC_VAULT_PATH` with the actual binary path and vault path, then install to the appropriate systemd directory.
### Planned options:
- `--immediate`: Perform sync after the command (e.g. `push`, `pull`, `put`, `rm`).
@@ -323,9 +515,9 @@ Note: `mirror` does not respect file deletions. If a file is deleted in storage,
Create default settings, apply a setup URI, then run one sync cycle.
```bash
npm run --silent cli -- init-settings /data/livesync-settings.json
printf '%s\n' "$SETUP_PASSPHRASE" | npm run --silent cli -- /data/vault --settings /data/livesync-settings.json setup "$SETUP_URI"
npm run --silent cli -- /data/vault --settings /data/livesync-settings.json sync
livesync-cli -- init-settings /data/livesync-settings.json
printf '%s\n' "$SETUP_PASSPHRASE" | livesync-cli -- /data/vault --settings /data/livesync-settings.json setup "$SETUP_URI"
livesync-cli -- /data/vault --settings /data/livesync-settings.json sync
```
### 2. Scripted import and export
@@ -333,8 +525,8 @@ npm run --silent cli -- /data/vault --settings /data/livesync-settings.json sync
Push local files into the database from automation, and pull them back for export or backup.
```bash
npm run --silent cli -- /data/vault --settings /data/livesync-settings.json push ./note.md notes/note.md
npm run --silent cli -- /data/vault --settings /data/livesync-settings.json pull notes/note.md ./exports/note.md
livesync-cli -- /data/vault --settings /data/livesync-settings.json push ./note.md notes/note.md
livesync-cli -- /data/vault --settings /data/livesync-settings.json pull notes/note.md ./exports/note.md
```
### 3. Revision inspection and restore
@@ -342,9 +534,9 @@ npm run --silent cli -- /data/vault --settings /data/livesync-settings.json pull
List metadata, find an older revision, then restore it by content (`cat-rev`) or file output (`pull-rev`).
```bash
npm run --silent cli -- /data/vault --settings /data/livesync-settings.json info notes/note.md
npm run --silent cli -- /data/vault --settings /data/livesync-settings.json cat-rev notes/note.md 3-abcdef
npm run --silent cli -- /data/vault --settings /data/livesync-settings.json pull-rev notes/note.md ./restore/note.old.md 3-abcdef
livesync-cli -- /data/vault --settings /data/livesync-settings.json info notes/note.md
livesync-cli -- /data/vault --settings /data/livesync-settings.json cat-rev notes/note.md 3-abcdef
livesync-cli -- /data/vault --settings /data/livesync-settings.json pull-rev notes/note.md ./restore/note.old.md 3-abcdef
```
### 4. Conflict and cleanup workflow
@@ -352,9 +544,9 @@ npm run --silent cli -- /data/vault --settings /data/livesync-settings.json pull
Inspect conflicted revisions, resolve by keeping one revision, then delete obsolete files.
```bash
npm run --silent cli -- /data/vault --settings /data/livesync-settings.json info notes/note.md
npm run --silent cli -- /data/vault --settings /data/livesync-settings.json resolve notes/note.md 3-abcdef
npm run --silent cli -- /data/vault --settings /data/livesync-settings.json rm notes/obsolete.md
livesync-cli -- /data/vault --settings /data/livesync-settings.json info notes/note.md
livesync-cli -- /data/vault --settings /data/livesync-settings.json resolve notes/note.md 3-abcdef
livesync-cli -- /data/vault --settings /data/livesync-settings.json rm notes/obsolete.md
```
### 5. CI smoke test for content round-trip
@@ -362,8 +554,8 @@ npm run --silent cli -- /data/vault --settings /data/livesync-settings.json rm n
Validate that `put`/`cat` is behaving as expected in a pipeline.
```bash
echo "hello-ci" | npm run --silent cli -- /data/vault --settings /data/livesync-settings.json put ci/test.md
npm run --silent cli -- /data/vault --settings /data/livesync-settings.json cat ci/test.md
echo "hello-ci" | livesync-cli -- /data/vault --settings /data/livesync-settings.json put ci/test.md
livesync-cli -- /data/vault --settings /data/livesync-settings.json cat ci/test.md
```
## Development

View File

@@ -39,12 +39,6 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
async getAbstractFileByPath(p: FilePath | string): Promise<NodeFile | null> {
const pathStr = this.normalisePath(p);
const cached = this.fileCache.get(pathStr);
if (cached) {
return cached;
}
return await this.refreshFile(pathStr);
}
@@ -104,14 +98,15 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
path: pathStr as FilePath,
stat: {
size: stat.size,
mtime: stat.mtimeMs,
ctime: stat.ctimeMs,
mtime: Math.floor(stat.mtimeMs),
ctime: Math.floor(stat.ctimeMs),
type: "file",
},
};
this.fileCache.set(pathStr, file);
return file;
} catch {
// Evict so a deleted file is not returned by subsequent cache scans.
this.fileCache.delete(pathStr);
return null;
}
@@ -137,8 +132,8 @@ export class NodeFileSystemAdapter implements IFileSystemAdapter<NodeFile, NodeF
path: entryRelativePath as FilePath,
stat: {
size: stat.size,
mtime: stat.mtimeMs,
ctime: stat.ctimeMs,
mtime: Math.floor(stat.mtimeMs),
ctime: Math.floor(stat.ctimeMs),
type: "file",
},
};

View File

@@ -28,8 +28,8 @@ export class NodeStorageAdapter implements IStorageAdapter<NodeStat> {
const stat = await fs.stat(this.resolvePath(p));
return {
size: stat.size,
mtime: stat.mtimeMs,
ctime: stat.ctimeMs,
mtime: Math.floor(stat.mtimeMs),
ctime: Math.floor(stat.ctimeMs),
type: stat.isDirectory() ? "folder" : "file",
};
} catch {

View File

@@ -15,7 +15,12 @@ export class NodeVaultAdapter implements IVaultAdapter<NodeFile> {
}
async read(file: NodeFile): Promise<string> {
return await fs.readFile(this.resolvePath(file.path), "utf-8");
const content = await fs.readFile(this.resolvePath(file.path), "utf-8");
// Correct stale stat.size — chokidar stats may be from a poll before the final write.
// The downstream document integrity check compares stat.size to content length, so
// they must agree or other clients reject the file as corrupted.
file.stat.size = Buffer.byteLength(content, "utf-8");
return content;
}
async cachedRead(file: NodeFile): Promise<string> {
@@ -25,6 +30,8 @@ export class NodeVaultAdapter implements IVaultAdapter<NodeFile> {
async readBinary(file: NodeFile): Promise<ArrayBuffer> {
const buffer = await fs.readFile(this.resolvePath(file.path));
// Same correction as read() — ensure stat.size matches actual byte length.
file.stat.size = buffer.length;
return buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength) as ArrayBuffer;
}
@@ -66,8 +73,8 @@ export class NodeVaultAdapter implements IVaultAdapter<NodeFile> {
path: p as any,
stat: {
size: stat.size,
mtime: stat.mtimeMs,
ctime: stat.ctimeMs,
mtime: Math.floor(stat.mtimeMs),
ctime: Math.floor(stat.ctimeMs),
type: "file",
},
};
@@ -89,8 +96,8 @@ export class NodeVaultAdapter implements IVaultAdapter<NodeFile> {
path: p as any,
stat: {
size: stat.size,
mtime: stat.mtimeMs,
ctime: stat.ctimeMs,
mtime: Math.floor(stat.mtimeMs),
ctime: Math.floor(stat.ctimeMs),
type: "file",
},
};

View File

@@ -0,0 +1,312 @@
import { describe, expect, it, vi, beforeEach, afterEach } from "vitest";
import { runCommand } from "./runCommand";
import type { CLIOptions } from "./types";
// Mock performFullScan so daemon tests don't require a real CouchDB connection.
vi.mock("@lib/serviceFeatures/offlineScanner", () => ({
performFullScan: vi.fn(async () => true),
}));
// Mock UnresolvedErrorManager to avoid event-hub side effects.
vi.mock("@lib/services/base/UnresolvedErrorManager", () => ({
UnresolvedErrorManager: class UnresolvedErrorManager {
showError() {}
clearError() {}
clearErrors() {}
},
}));
import * as offlineScanner from "@lib/serviceFeatures/offlineScanner";
function createCoreMock() {
return {
services: {
control: {
activated: Promise.resolve(),
applySettings: vi.fn(async () => {}),
},
setting: {
applyPartial: vi.fn(async () => {}),
currentSettings: vi.fn(() => ({ liveSync: true, syncOnStart: false })),
},
replication: {
replicate: vi.fn(async () => true),
},
appLifecycle: {
onUnload: {
addHandler: vi.fn(),
},
},
},
serviceModules: {
fileHandler: {
dbToStorage: vi.fn(async () => true),
storeFileToDB: vi.fn(async () => true),
},
storageAccess: {
readFileAuto: vi.fn(async () => ""),
writeFileAuto: vi.fn(async () => {}),
},
databaseFileAccess: {
fetch: vi.fn(async () => undefined),
},
},
} as any;
}
function makeDaemonOptions(interval?: number): CLIOptions {
return {
command: "daemon",
commandArgs: [],
databasePath: "/tmp/vault",
verbose: false,
force: false,
interval,
};
}
const baseContext = {
vaultPath: "/tmp/vault",
settingsPath: "/tmp/vault/.livesync/settings.json",
originalSyncSettings: {
liveSync: true,
syncOnStart: false,
periodicReplication: false,
syncOnSave: false,
syncOnEditorSave: false,
syncOnFileOpen: false,
syncAfterMerge: false,
},
} as any;
describe("daemon command", () => {
beforeEach(() => {
vi.restoreAllMocks();
vi.useFakeTimers();
});
afterEach(() => {
vi.useRealTimers();
});
it("calls performFullScan during startup", async () => {
const core = createCoreMock();
vi.mocked(offlineScanner.performFullScan).mockResolvedValue(true);
await runCommand(makeDaemonOptions(), { ...baseContext, core });
expect(offlineScanner.performFullScan).toHaveBeenCalledTimes(1);
});
it("returns false when performFullScan fails", async () => {
const core = createCoreMock();
vi.mocked(offlineScanner.performFullScan).mockResolvedValue(false);
const result = await runCommand(makeDaemonOptions(), { ...baseContext, core });
expect(result).toBe(false);
});
it("polling mode: calls setTimeout when interval option is set", async () => {
const core = createCoreMock();
vi.mocked(offlineScanner.performFullScan).mockResolvedValue(true);
const setTimeoutSpy = vi.spyOn(globalThis, "setTimeout");
await runCommand(makeDaemonOptions(30), { ...baseContext, core });
expect(setTimeoutSpy).toHaveBeenCalledTimes(1);
// Interval should be in milliseconds (30s → 30000ms)
expect(setTimeoutSpy).toHaveBeenCalledWith(expect.any(Function), 30000);
});
it("polling mode: applies settings with suspendFileWatching=false before setting interval", async () => {
const core = createCoreMock();
vi.mocked(offlineScanner.performFullScan).mockResolvedValue(true);
await runCommand(makeDaemonOptions(10), { ...baseContext, core });
expect(core.services.setting.applyPartial).toHaveBeenCalledWith(
expect.objectContaining({ suspendFileWatching: false }),
true
);
expect(core.services.control.applySettings).toHaveBeenCalledTimes(1);
});
it("liveSync mode: calls applyPartial and applySettings", async () => {
const core = createCoreMock();
vi.mocked(offlineScanner.performFullScan).mockResolvedValue(true);
await runCommand(makeDaemonOptions(), { ...baseContext, core });
expect(core.services.setting.applyPartial).toHaveBeenCalledWith(
expect.objectContaining({
...baseContext.originalSyncSettings,
suspendFileWatching: false,
}),
true
);
expect(core.services.control.applySettings).toHaveBeenCalledTimes(1);
});
it("liveSync mode: logs warning when both liveSync and syncOnStart are false", async () => {
const core = createCoreMock();
core.services.setting.currentSettings = vi.fn(() => ({
liveSync: false,
syncOnStart: false,
}));
vi.mocked(offlineScanner.performFullScan).mockResolvedValue(true);
const consoleSpy = vi.spyOn(console, "error").mockImplementation(() => {});
const result = await runCommand(makeDaemonOptions(), { ...baseContext, core });
expect(result).toBe(true);
const warningCalls = consoleSpy.mock.calls.filter(
(args) => typeof args[0] === "string" && args[0].includes("liveSync and syncOnStart are both disabled")
);
expect(warningCalls.length).toBeGreaterThan(0);
});
it("liveSync mode: no warning when liveSync is true", async () => {
const core = createCoreMock();
core.services.setting.currentSettings = vi.fn(() => ({
liveSync: true,
syncOnStart: false,
}));
vi.mocked(offlineScanner.performFullScan).mockResolvedValue(true);
const consoleSpy = vi.spyOn(console, "error").mockImplementation(() => {});
await runCommand(makeDaemonOptions(), { ...baseContext, core });
const warningCalls = consoleSpy.mock.calls.filter(
(args) => typeof args[0] === "string" && args[0].includes("liveSync and syncOnStart are both disabled")
);
expect(warningCalls.length).toBe(0);
});
it("calls replicate before performFullScan", async () => {
const core = createCoreMock();
const callOrder: string[] = [];
core.services.replication.replicate = vi.fn(async () => {
callOrder.push("replicate");
return true;
});
vi.mocked(offlineScanner.performFullScan).mockImplementation(async () => {
callOrder.push("performFullScan");
return true;
});
await runCommand(makeDaemonOptions(), { ...baseContext, core });
expect(callOrder).toEqual(["replicate", "performFullScan"]);
});
it("returns false when initial replication fails", async () => {
const core = createCoreMock();
core.services.replication.replicate = vi.fn(async () => false);
vi.mocked(offlineScanner.performFullScan).mockClear();
const result = await runCommand(makeDaemonOptions(), { ...baseContext, core });
expect(result).toBe(false);
// performFullScan should NOT have been called
expect(offlineScanner.performFullScan).not.toHaveBeenCalled();
});
it("polling mode: registers onUnload handler that clears timeout", async () => {
const core = createCoreMock();
vi.mocked(offlineScanner.performFullScan).mockResolvedValue(true);
await runCommand(makeDaemonOptions(10), { ...baseContext, core });
// onUnload handler should have been registered
expect(core.services.appLifecycle.onUnload.addHandler).toHaveBeenCalledTimes(1);
const handler = core.services.appLifecycle.onUnload.addHandler.mock.calls[0][0];
// Get the timeout ID that was created
const clearTimeoutSpy = vi.spyOn(globalThis, "clearTimeout");
await handler();
expect(clearTimeoutSpy).toHaveBeenCalledTimes(1);
});
it("polling backoff: interval escalates on failure, caps at 300000ms, then halves on recovery", async () => {
const core = createCoreMock();
vi.mocked(offlineScanner.performFullScan).mockResolvedValue(true);
vi.spyOn(console, "error").mockImplementation(() => {});
// startup replicate (call 1) succeeds; poll calls 27 fail; call 8 succeeds.
let callCount = 0;
core.services.replication.replicate = vi.fn(async () => {
callCount++;
if (callCount === 1) return true; // initial startup replicate
if (callCount <= 7) throw new Error("network failure");
return true; // recovery
});
const baseMs = 30 * 1000;
const setTimeoutSpy = vi.spyOn(globalThis, "setTimeout");
await runCommand(makeDaemonOptions(30), { ...baseContext, core });
// After runCommand returns the first setTimeout has been scheduled.
// setTimeoutSpy.mock.calls[0] is the initial schedule (baseMs).
expect(setTimeoutSpy.mock.calls[0][1]).toBe(baseMs);
// Advance through 6 failure polls. After each failure the next setTimeout
// should be scheduled with a larger (or capped) interval.
// formula: min(base * 2^n, 300000). base=30000ms.
// failure 1: 30000*2=60000, failure 2: 30000*4=120000,
// failure 3: 30000*8=240000, failure 4: 30000*16=480000→capped, 5→cap, 6→cap
const expectedIntervals = [
baseMs * 2, // after failure 1: 60000
baseMs * 4, // after failure 2: 120000
baseMs * 8, // after failure 3: 240000
300_000, // after failure 4 (would be 480000, capped)
300_000, // after failure 5 (cap)
300_000, // after failure 6 (cap)
];
for (const expected of expectedIntervals) {
const prevCallCount = setTimeoutSpy.mock.calls.length;
await vi.advanceTimersByTimeAsync(setTimeoutSpy.mock.calls[prevCallCount - 1][1] as number);
const newCallCount = setTimeoutSpy.mock.calls.length;
expect(newCallCount).toBeGreaterThan(prevCallCount);
expect(setTimeoutSpy.mock.calls[newCallCount - 1][1]).toBe(expected);
}
// Now trigger the success poll — interval should halve each time toward base.
// After failure 6, consecutiveFailures=6, currentIntervalMs=300000.
// On success: consecutiveFailures=5, currentIntervalMs=150000.
const prevCallCount = setTimeoutSpy.mock.calls.length;
await vi.advanceTimersByTimeAsync(setTimeoutSpy.mock.calls[prevCallCount - 1][1] as number);
const afterSuccessCallCount = setTimeoutSpy.mock.calls.length;
expect(afterSuccessCallCount).toBeGreaterThan(prevCallCount);
// The interval after one success should be halved (300000 / 2 = 150000).
expect(setTimeoutSpy.mock.calls[afterSuccessCallCount - 1][1]).toBe(150_000);
});
it("polling error handling: replicate rejection is caught and console.error is called", async () => {
const core = createCoreMock();
vi.mocked(offlineScanner.performFullScan).mockResolvedValue(true);
const consoleSpy = vi.spyOn(console, "error").mockImplementation(() => {});
// Make replicate succeed on the initial call (startup), then fail on the poll.
let callCount = 0;
core.services.replication.replicate = vi.fn(async () => {
callCount++;
if (callCount === 1) return true; // startup replicate
throw new Error("network failure");
});
const intervalMs = 30 * 1000;
await runCommand(makeDaemonOptions(30), { ...baseContext, core });
// Advance time to trigger the first poll callback and flush its async work.
await vi.advanceTimersByTimeAsync(intervalMs);
// No unhandled rejection — the error was caught internally.
const errorCalls = consoleSpy.mock.calls.filter(
(args) => typeof args[0] === "string" && args[0].includes("Poll error")
);
expect(errorCalls.length).toBeGreaterThan(0);
});
});

View File

@@ -32,10 +32,15 @@ function validateP2PSettings(core: LiveSyncBaseCore<ServiceContext, any>) {
settings.P2P_IsHeadless = true;
}
function createReplicator(core: LiveSyncBaseCore<ServiceContext, any>): LiveSyncTrysteroReplicator {
async function createReplicator(core: LiveSyncBaseCore<ServiceContext, any>): Promise<LiveSyncTrysteroReplicator> {
validateP2PSettings(core);
const replicator = new LiveSyncTrysteroReplicator({ services: core.services });
addP2PEventHandlers(replicator);
const replicator = await core.services.replicator.getNewReplicator();
if (!replicator) {
throw new Error("Failed to create replicator instance. Ensure P2P is enabled in settings.");
}
if (!(replicator instanceof LiveSyncTrysteroReplicator)) {
throw new Error("Unexpected replicator type. Expected LiveSyncTrysteroReplicator.");
}
return replicator;
}
@@ -49,7 +54,7 @@ export async function collectPeers(
core: LiveSyncBaseCore<ServiceContext, any>,
timeoutSec: number
): Promise<CLIP2PPeer[]> {
const replicator = createReplicator(core);
const replicator = await createReplicator(core);
await replicator.open();
try {
await delay(timeoutSec * 1000);
@@ -79,7 +84,7 @@ export async function syncWithPeer(
peerToken: string,
timeoutSec: number
): Promise<CLIP2PPeer> {
const replicator = createReplicator(core);
const replicator = await createReplicator(core);
await replicator.open();
try {
const timeoutMs = timeoutSec * 1000;
@@ -115,7 +120,7 @@ export async function syncWithPeer(
}
export async function openP2PHost(core: LiveSyncBaseCore<ServiceContext, any>): Promise<LiveSyncTrysteroReplicator> {
const replicator = createReplicator(core);
const replicator = await createReplicator(core);
await replicator.open();
return replicator;
}

View File

@@ -3,18 +3,124 @@ import * as path from "path";
import { decodeSettingsFromSetupURI } from "@lib/API/processSetting";
import { configURIBase } from "@lib/common/models/shared.const";
import { DEFAULT_SETTINGS, type FilePathWithPrefix, type ObsidianLiveSyncSettings } from "@lib/common/types";
import { ConnectionStringParser } from "@lib/common/ConnectionString";
import { activateRemoteConfiguration, createRemoteConfigurationId } from "@lib/serviceFeatures/remoteConfig";
import { stripAllPrefixes } from "@lib/string_and_binary/path";
import type { CLICommandContext, CLIOptions } from "./types";
import { promptForPassphrase, readStdinAsUtf8, toArrayBuffer, toVaultRelativePath } from "./utils";
import { promptForPassphrase, readStdinAsUtf8, toArrayBuffer, toDatabaseRelativePath } from "./utils";
import { collectPeers, openP2PHost, parseTimeoutSeconds, syncWithPeer } from "./p2p";
import { performFullScan } from "@lib/serviceFeatures/offlineScanner";
import { UnresolvedErrorManager } from "@lib/services/base/UnresolvedErrorManager";
function redactConnectionString(uri: string): string {
return uri.replace(/\/\/([^@/]+)@/u, "//***@");
}
export async function runCommand(options: CLIOptions, context: CLICommandContext): Promise<boolean> {
const { vaultPath, core, settingsPath } = context;
const { databasePath, core, settingsPath } = context;
await core.services.control.activated;
if (options.command === "daemon") {
const log = (msg: unknown) => console.error(`[Daemon] ${msg}`);
// Skip the config mismatch dialog — the daemon cannot resolve it interactively
// and the default "Dismiss" action would block replication. The daemon should
// accept whatever configuration the remote has.
await core.services.setting.applyPartial({ disableCheckingConfigMismatch: true }, true);
// 1. Replicate CouchDB → local PouchDB so the mirror scan has content to work with.
log("Replicating from CouchDB...");
const replResult = await core.services.replication.replicate(true);
if (!replResult) {
console.error("[Daemon] Initial CouchDB replication failed, cannot continue");
return false;
}
log("CouchDB replication complete");
// 2. Mirror scan to reconcile PouchDB ↔ local filesystem.
const errorManager = new UnresolvedErrorManager(core.services.appLifecycle);
log("Running mirror scan...");
const scanOk = await performFullScan(core as any, log, errorManager, false, true);
if (!scanOk) {
console.error("[Daemon] Mirror scan failed, cannot continue");
return false;
}
log("Mirror scan complete");
// 3. Re-enable sync.
const restoreSyncSettings = async () => {
await core.services.setting.applyPartial(
{
...context.originalSyncSettings,
suspendFileWatching: false,
},
true
);
// applySettings fires the full lifecycle: onSuspending → onResumed.
// ModuleReplicatorCouchDB starts continuous replication on onResumed
// via fireAndForget.
await core.services.control.applySettings();
// Lifecycle events (onSuspending) may re-enable suspension flags.
// Clear them explicitly after the lifecycle completes. applyPartial
// with true is a direct store write — it does not re-trigger lifecycle.
await core.services.setting.applyPartial(
{
suspendFileWatching: false,
suspendParseReplicationResult: false,
},
true
);
};
if (options.interval) {
log(`Polling mode: syncing every ${options.interval}s`);
await restoreSyncSettings();
const baseIntervalMs = options.interval * 1000;
let currentIntervalMs = baseIntervalMs;
let consecutiveFailures = 0;
const maxIntervalMs = 5 * 60 * 1000; // 5 minutes cap
const poll = async () => {
try {
await core.services.replication.replicate(true);
if (consecutiveFailures > 0) {
consecutiveFailures--;
currentIntervalMs = Math.max(currentIntervalMs / 2, baseIntervalMs);
log(`Replication recovered`);
}
} catch (err) {
consecutiveFailures++;
currentIntervalMs = Math.min(baseIntervalMs * Math.pow(2, consecutiveFailures), maxIntervalMs);
console.error(`[Daemon] Poll error (${consecutiveFailures} consecutive):`, err);
if (consecutiveFailures >= 5) {
console.error(
`[Daemon] Warning: ${consecutiveFailures} consecutive failures, backing off to ${Math.round(currentIntervalMs / 1000)}s`
);
}
}
pollTimer = setTimeout(poll, currentIntervalMs);
};
let pollTimer: ReturnType<typeof setTimeout> = setTimeout(poll, currentIntervalMs);
core.services.appLifecycle.onUnload.addHandler(async () => {
clearTimeout(pollTimer);
return true;
});
} else {
log("LiveSync mode: restoring sync settings and starting _changes feed");
await restoreSyncSettings();
// The applySettings() lifecycle fires onResumed → ModuleReplicatorCouchDB which
// starts continuous replication via fireAndForget(openReplication). Don't call
// openReplication directly — it races with the handler and causes dedup/termination.
log("LiveSync active");
const currentSettings = core.services.setting.currentSettings();
if (!currentSettings.liveSync && !currentSettings.syncOnStart) {
console.error(
"[Daemon] Warning: liveSync and syncOnStart are both disabled in settings. " +
"No sync will occur. Set liveSync=true in your settings file for continuous sync, " +
"or use --interval for polling mode."
);
}
}
return true;
}
@@ -77,16 +183,16 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
throw new Error("push requires two arguments: <src> <dst>");
}
const sourcePath = path.resolve(options.commandArgs[0]);
const destinationVaultPath = toVaultRelativePath(options.commandArgs[1], vaultPath);
const destinationDatabasePath = toDatabaseRelativePath(options.commandArgs[1], databasePath);
const sourceData = await fs.readFile(sourcePath);
const sourceStat = await fs.stat(sourcePath);
console.log(`[Command] push ${sourcePath} -> ${destinationVaultPath}`);
console.log(`[Command] push ${sourcePath} -> ${destinationDatabasePath}`);
await core.serviceModules.storageAccess.writeFileAuto(destinationVaultPath, toArrayBuffer(sourceData), {
mtime: sourceStat.mtimeMs,
ctime: sourceStat.ctimeMs,
await core.serviceModules.storageAccess.writeFileAuto(destinationDatabasePath, toArrayBuffer(sourceData), {
mtime: Math.floor(sourceStat.mtimeMs),
ctime: Math.floor(sourceStat.ctimeMs),
});
const destinationPathWithPrefix = destinationVaultPath as FilePathWithPrefix;
const destinationPathWithPrefix = destinationDatabasePath as FilePathWithPrefix;
const stored = await core.serviceModules.fileHandler.storeFileToDB(destinationPathWithPrefix, true);
return stored;
}
@@ -95,16 +201,16 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
if (options.commandArgs.length < 2) {
throw new Error("pull requires two arguments: <src> <dst>");
}
const sourceVaultPath = toVaultRelativePath(options.commandArgs[0], vaultPath);
const sourceDatabasePath = toDatabaseRelativePath(options.commandArgs[0], databasePath);
const destinationPath = path.resolve(options.commandArgs[1]);
console.log(`[Command] pull ${sourceVaultPath} -> ${destinationPath}`);
console.log(`[Command] pull ${sourceDatabasePath} -> ${destinationPath}`);
const sourcePathWithPrefix = sourceVaultPath as FilePathWithPrefix;
const sourcePathWithPrefix = sourceDatabasePath as FilePathWithPrefix;
const restored = await core.serviceModules.fileHandler.dbToStorage(sourcePathWithPrefix, null, true);
if (!restored) {
return false;
}
const data = await core.serviceModules.storageAccess.readFileAuto(sourceVaultPath);
const data = await core.serviceModules.storageAccess.readFileAuto(sourceDatabasePath);
await fs.mkdir(path.dirname(destinationPath), { recursive: true });
if (typeof data === "string") {
await fs.writeFile(destinationPath, data, "utf-8");
@@ -118,16 +224,16 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
if (options.commandArgs.length < 3) {
throw new Error("pull-rev requires three arguments: <src> <dst> <rev>");
}
const sourceVaultPath = toVaultRelativePath(options.commandArgs[0], vaultPath);
const sourceDatabasePath = toDatabaseRelativePath(options.commandArgs[0], databasePath);
const destinationPath = path.resolve(options.commandArgs[1]);
const rev = options.commandArgs[2].trim();
if (!rev) {
throw new Error("pull-rev requires a non-empty revision");
}
console.log(`[Command] pull-rev ${sourceVaultPath}@${rev} -> ${destinationPath}`);
console.log(`[Command] pull-rev ${sourceDatabasePath}@${rev} -> ${destinationPath}`);
const source = await core.serviceModules.databaseFileAccess.fetch(
sourceVaultPath as FilePathWithPrefix,
sourceDatabasePath as FilePathWithPrefix,
rev,
true
);
@@ -175,11 +281,11 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
if (options.commandArgs.length < 1) {
throw new Error("put requires one argument: <dst>");
}
const destinationVaultPath = toVaultRelativePath(options.commandArgs[0], vaultPath);
const destinationDatabasePath = toDatabaseRelativePath(options.commandArgs[0], databasePath);
const content = await readStdinAsUtf8();
console.log(`[Command] put stdin -> ${destinationVaultPath}`);
console.log(`[Command] put stdin -> ${destinationDatabasePath}`);
return await core.serviceModules.databaseFileAccess.storeContent(
destinationVaultPath as FilePathWithPrefix,
destinationDatabasePath as FilePathWithPrefix,
content
);
}
@@ -188,10 +294,10 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
if (options.commandArgs.length < 1) {
throw new Error("cat requires one argument: <src>");
}
const sourceVaultPath = toVaultRelativePath(options.commandArgs[0], vaultPath);
console.error(`[Command] cat ${sourceVaultPath}`);
const sourceDatabasePath = toDatabaseRelativePath(options.commandArgs[0], databasePath);
console.error(`[Command] cat ${sourceDatabasePath}`);
const source = await core.serviceModules.databaseFileAccess.fetch(
sourceVaultPath as FilePathWithPrefix,
sourceDatabasePath as FilePathWithPrefix,
undefined,
true
);
@@ -212,14 +318,14 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
if (options.commandArgs.length < 2) {
throw new Error("cat-rev requires two arguments: <src> <rev>");
}
const sourceVaultPath = toVaultRelativePath(options.commandArgs[0], vaultPath);
const sourceDatabasePath = toDatabaseRelativePath(options.commandArgs[0], databasePath);
const rev = options.commandArgs[1].trim();
if (!rev) {
throw new Error("cat-rev requires a non-empty revision");
}
console.error(`[Command] cat-rev ${sourceVaultPath} @ ${rev}`);
console.error(`[Command] cat-rev ${sourceDatabasePath} @ ${rev}`);
const source = await core.serviceModules.databaseFileAccess.fetch(
sourceVaultPath as FilePathWithPrefix,
sourceDatabasePath as FilePathWithPrefix,
rev,
true
);
@@ -239,7 +345,7 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
if (options.command === "ls") {
const prefix =
options.commandArgs.length > 0 && options.commandArgs[0].trim() !== ""
? toVaultRelativePath(options.commandArgs[0], vaultPath)
? toDatabaseRelativePath(options.commandArgs[0], databasePath)
: "";
const rows: { path: string; line: string }[] = [];
@@ -261,6 +367,8 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
rows.sort((a, b) => a.path.localeCompare(b.path));
if (rows.length > 0) {
process.stdout.write(rows.map((e) => e.line).join("\n") + "\n");
} else {
process.stderr.write("[Info] No documents found in the local database.\n");
}
return true;
}
@@ -269,7 +377,7 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
if (options.commandArgs.length < 1) {
throw new Error("info requires one argument: <path>");
}
const targetPath = toVaultRelativePath(options.commandArgs[0], vaultPath);
const targetPath = toDatabaseRelativePath(options.commandArgs[0], databasePath);
for await (const doc of core.services.database.localDatabase.findAllNormalDocs({ conflicts: true })) {
if (doc._deleted || doc.deleted) continue;
@@ -313,7 +421,7 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
if (options.commandArgs.length < 1) {
throw new Error("rm requires one argument: <path>");
}
const targetPath = toVaultRelativePath(options.commandArgs[0], vaultPath);
const targetPath = toDatabaseRelativePath(options.commandArgs[0], databasePath);
console.error(`[Command] rm ${targetPath}`);
return await core.serviceModules.databaseFileAccess.delete(targetPath as FilePathWithPrefix);
}
@@ -322,7 +430,7 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
if (options.commandArgs.length < 2) {
throw new Error("resolve requires two arguments: <path> <revision-to-keep>");
}
const targetPath = toVaultRelativePath(options.commandArgs[0], vaultPath) as FilePathWithPrefix;
const targetPath = toDatabaseRelativePath(options.commandArgs[0], databasePath) as FilePathWithPrefix;
const revisionToKeep = options.commandArgs[1].trim();
if (revisionToKeep === "") {
throw new Error("resolve requires a non-empty revision-to-keep");
@@ -367,5 +475,206 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
return await performFullScan(core as any, log, errorManager, false, true);
}
if (options.command === "remote-add") {
if (options.commandArgs.length < 2) {
throw new Error("remote-add requires two arguments: <name> <connstr>");
}
const name = options.commandArgs[0].trim();
const connectionString = options.commandArgs[1].trim();
if (!name) {
throw new Error("remote-add requires a non-empty name");
}
if (!connectionString) {
throw new Error("remote-add requires a non-empty connection string");
}
const parsed = ConnectionStringParser.parse(connectionString);
const canonicalUri = ConnectionStringParser.serialize(parsed);
const id = createRemoteConfigurationId();
let activated = false;
await core.services.setting.updateSettings((currentSettings) => {
currentSettings.remoteConfigurations ||= {};
currentSettings.remoteConfigurations[id] = {
id,
name,
uri: canonicalUri,
isEncrypted: false,
};
if (!currentSettings.activeConfigurationId) {
currentSettings.activeConfigurationId = id;
const applied = activateRemoteConfiguration(currentSettings, id);
activated = applied !== false;
}
return currentSettings;
}, true);
if (activated) {
await core.services.control.applySettings();
}
process.stdout.write(`${id}\t${name}\t${redactConnectionString(canonicalUri)}\n`);
return true;
}
if (options.command === "remote-rm") {
if (options.commandArgs.length < 1) {
throw new Error("remote-rm requires one argument: <remote-id>");
}
const id = options.commandArgs[0].trim();
if (!id) {
throw new Error("remote-rm requires a non-empty remote-id");
}
const current = core.services.setting.currentSettings();
if (!current.remoteConfigurations?.[id]) {
process.stderr.write(`[Info] Remote configuration not found: ${id}\n`);
return false;
}
let switchedActive = false;
await core.services.setting.updateSettings((currentSettings) => {
const configs = currentSettings.remoteConfigurations || {};
delete configs[id];
currentSettings.remoteConfigurations = configs;
if (currentSettings.activeConfigurationId === id) {
const nextActiveId = Object.keys(configs)[0] || "";
currentSettings.activeConfigurationId = nextActiveId;
switchedActive = nextActiveId !== "";
if (nextActiveId !== "") {
activateRemoteConfiguration(currentSettings, nextActiveId);
}
}
if (currentSettings.P2P_ActiveRemoteConfigurationId === id) {
currentSettings.P2P_ActiveRemoteConfigurationId = "";
}
return currentSettings;
}, true);
if (switchedActive) {
await core.services.control.applySettings();
}
console.error(`[Command] remote-rm ${id}`);
return true;
}
if (options.command === "remote-ls") {
const settings = core.services.setting.currentSettings();
const configs = Object.values(settings.remoteConfigurations || {});
configs.sort((a, b) => a.name.localeCompare(b.name));
if (configs.length === 0) {
process.stderr.write("[Info] No remote configurations found.\n");
return true;
}
const lines = configs.map((config) => {
const status = config.id === settings.activeConfigurationId ? "active" : "inactive";
return `${config.id}\t${config.name}\t${status}\t${redactConnectionString(config.uri)}`;
});
process.stdout.write(lines.join("\n") + "\n");
return true;
}
if (options.command === "remote-export") {
if (options.commandArgs.length < 1) {
throw new Error("remote-export requires one argument: <remote-id>");
}
const id = options.commandArgs[0].trim();
if (!id) {
throw new Error("remote-export requires a non-empty remote-id");
}
const config = core.services.setting.currentSettings().remoteConfigurations?.[id];
if (!config) {
process.stderr.write(`[Info] Remote configuration not found: ${id}\n`);
return false;
}
process.stdout.write(`${config.uri}\n`);
return true;
}
if (options.command === "remote-set") {
if (options.commandArgs.length < 2) {
throw new Error("remote-set requires two arguments: <remote-id> <connstr>");
}
const id = options.commandArgs[0].trim();
const connectionString = options.commandArgs[1].trim();
if (!id) {
throw new Error("remote-set requires a non-empty remote-id");
}
if (!connectionString) {
throw new Error("remote-set requires a non-empty connection string");
}
const parsed = ConnectionStringParser.parse(connectionString);
const canonicalUri = ConnectionStringParser.serialize(parsed);
let switchedActive = false;
await core.services.setting.updateSettings((currentSettings) => {
const config = currentSettings.remoteConfigurations?.[id];
if (!config) {
return currentSettings;
}
config.uri = canonicalUri;
if (currentSettings.activeConfigurationId === id) {
const activated = activateRemoteConfiguration(currentSettings, id);
switchedActive = activated !== false;
if (activated) {
return activated;
}
}
return currentSettings;
}, true);
const updated = core.services.setting.currentSettings().remoteConfigurations?.[id];
if (!updated) {
process.stderr.write(`[Info] Remote configuration not found: ${id}\n`);
return false;
}
if (switchedActive) {
await core.services.control.applySettings();
}
console.error(`[Command] remote-set ${id}`);
return true;
}
if (options.command === "remote-activate") {
if (options.commandArgs.length < 1) {
throw new Error("remote-activate requires one argument: <remote-id>");
}
const id = options.commandArgs[0].trim();
if (!id) {
throw new Error("remote-activate requires a non-empty remote-id");
}
let switched = false;
await core.services.setting.updateSettings((currentSettings) => {
const activated = activateRemoteConfiguration(currentSettings, id);
if (activated) {
switched = true;
return activated;
}
return currentSettings;
}, true);
if (!switched) {
process.stderr.write(`[Info] Failed to activate remote configuration: ${id}\n`);
return false;
}
await core.services.control.applySettings();
console.error(`[Command] remote-activate ${id}`);
return true;
}
throw new Error(`Unsupported command: ${options.command}`);
}

View File

@@ -1,12 +1,19 @@
import * as processSetting from "@lib/API/processSetting";
import { ConnectionStringParser } from "@lib/common/ConnectionString";
import { configURIBase } from "@lib/common/models/shared.const";
import { DEFAULT_SETTINGS } from "@lib/common/types";
import { DEFAULT_SETTINGS, REMOTE_COUCHDB, REMOTE_MINIO, REMOTE_P2P } from "@lib/common/types";
import { describe, expect, it, vi, beforeEach, afterEach } from "vitest";
import { runCommand } from "./runCommand";
import type { CLIOptions } from "./types";
import * as commandUtils from "./utils";
function createCoreMock() {
const liveSettings = {
...DEFAULT_SETTINGS,
remoteConfigurations: {},
activeConfigurationId: "",
P2P_ActiveRemoteConfigurationId: "",
} as any;
return {
services: {
control: {
@@ -16,6 +23,10 @@ function createCoreMock() {
setting: {
applyExternalSettings: vi.fn(async () => {}),
applyPartial: vi.fn(async () => {}),
currentSettings: vi.fn(() => liveSettings),
updateSettings: vi.fn(async (updater: any) => {
updater(liveSettings);
}),
},
},
serviceModules: {
@@ -56,9 +67,118 @@ async function createSetupURI(passphrase: string): Promise<string> {
return await processSetting.encodeSettingsToSetupURI(settings, passphrase);
}
function captureStdout() {
const writes: string[] = [];
const spy = vi.spyOn(process.stdout, "write").mockImplementation((chunk: any) => {
writes.push(typeof chunk === "string" ? chunk : String(chunk));
return true;
});
return {
spy,
lines: () =>
writes
.join("")
.split("\n")
.map((e) => e.trim())
.filter((e) => e.length > 0),
};
}
function parseAddedRemoteIdFromLines(lines: string[]): string {
// remote-add prints: <id>\t<name>\t<redacted-connstr>
const last = lines.length > 0 ? lines[lines.length - 1] : "";
return last.split("\t")[0] || "";
}
type ProtocolFixture = {
protocol: string;
connectionString: string;
assertProjectedFields: (settings: any) => void;
};
const protocolFixtures: ProtocolFixture[] = [
{
protocol: "couchdb",
connectionString: ConnectionStringParser.serialize({
type: "couchdb",
settings: {
couchDB_URI: "https://db.example.com:5984",
couchDB_USER: "user1",
couchDB_PASSWORD: "pass1",
couchDB_DBNAME: "vault1",
couchDB_CustomHeaders: "",
useJWT: false,
jwtAlgorithm: "",
jwtKey: "",
jwtKid: "",
jwtSub: "",
jwtExpDuration: 5,
useRequestAPI: false,
},
}),
assertProjectedFields: (settings) => {
expect(settings.remoteType).toBe(REMOTE_COUCHDB);
expect(settings.couchDB_URI).toBe("https://db.example.com:5984");
expect(settings.couchDB_USER).toBe("user1");
expect(settings.couchDB_PASSWORD).toBe("pass1");
expect(settings.couchDB_DBNAME).toBe("vault1");
},
},
{
protocol: "s3",
connectionString: ConnectionStringParser.serialize({
type: "s3",
settings: {
accessKey: "ak",
secretKey: "sk",
endpoint: "https://s3.example.com",
bucket: "bucket-1",
region: "ap-northeast-1",
bucketPrefix: "vault/",
useCustomRequestHandler: true,
bucketCustomHeaders: "x-test:1",
forcePathStyle: false,
},
}),
assertProjectedFields: (settings) => {
expect(settings.remoteType).toBe(REMOTE_MINIO);
expect(settings.accessKey).toBe("ak");
expect(settings.secretKey).toBe("sk");
expect(settings.endpoint).toBe("https://s3.example.com");
expect(settings.bucket).toBe("bucket-1");
expect(settings.region).toBe("ap-northeast-1");
},
},
{
protocol: "p2p",
connectionString: ConnectionStringParser.serialize({
type: "p2p",
settings: {
P2P_Enabled: false,
P2P_roomID: "room-abc",
P2P_passphrase: "pass-123",
P2P_relays: "wss://relay.example",
P2P_AppID: "self-hosted-livesync",
P2P_AutoStart: true,
P2P_AutoBroadcast: false,
P2P_turnServers: "turn:turn.example:3478",
P2P_turnUsername: "turn-user",
P2P_turnCredential: "turn-pass",
},
}),
assertProjectedFields: (settings) => {
expect(settings.remoteType).toBe(REMOTE_P2P);
expect(settings.P2P_roomID).toBe("room-abc");
expect(settings.P2P_passphrase).toBe("pass-123");
expect(settings.P2P_relays).toBe("wss://relay.example");
expect(settings.P2P_AppID).toBe("self-hosted-livesync");
},
},
];
describe("runCommand abnormal cases", () => {
const context = {
vaultPath: "/tmp/vault",
databasePath: "/tmp/vault",
settingsPath: "/tmp/vault/.livesync/settings.json",
} as any;
@@ -202,4 +322,254 @@ describe("runCommand abnormal cases", () => {
expect(core.services.setting.applyExternalSettings).not.toHaveBeenCalled();
expect(core.services.control.applySettings).not.toHaveBeenCalled();
});
it("remote-add stores canonical URI and prints the created id", async () => {
const core = createCoreMock();
const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true);
const result = await runCommand(makeOptions("remote-add", ["my-remote", "sls+https://example.com/db"]), {
...context,
core,
});
expect(result).toBe(true);
const settings = core.services.setting.currentSettings();
const ids = Object.keys(settings.remoteConfigurations);
expect(ids.length).toBe(1);
expect(settings.remoteConfigurations[ids[0]].name).toBe("my-remote");
expect(settings.remoteConfigurations[ids[0]].uri).toContain("sls+https://example.com/db");
expect(settings.activeConfigurationId).toBe(ids[0]);
expect(stdout).toHaveBeenCalled();
});
it("remote-activate switches active remote and applies settings", async () => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://example.com/db1",
isEncrypted: false,
};
settings.remoteConfigurations.r2 = {
id: "r2",
name: "R2",
uri: "sls+https://example.com/db2",
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-activate", ["r2"]), {
...context,
core,
});
expect(result).toBe(true);
expect(settings.activeConfigurationId).toBe("r2");
expect(core.services.control.applySettings).toHaveBeenCalledTimes(1);
});
it("remote-rm removes active remote and promotes first remaining", async () => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://example.com/db1",
isEncrypted: false,
};
settings.remoteConfigurations.r2 = {
id: "r2",
name: "R2",
uri: "sls+https://example.com/db2",
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-rm", ["r1"]), {
...context,
core,
});
expect(result).toBe(true);
expect(settings.remoteConfigurations.r1).toBeUndefined();
expect(settings.activeConfigurationId).toBe("r2");
expect(core.services.control.applySettings).toHaveBeenCalledTimes(1);
});
it("remote-export prints the exact stored connection string", async () => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://example.com/db?db=vault",
isEncrypted: false,
};
const stdout = captureStdout();
const result = await runCommand(makeOptions("remote-export", ["r1"]), {
...context,
core,
});
expect(result).toBe(true);
const outLines = stdout.lines();
expect(outLines.length > 0 ? outLines[outLines.length - 1] : "").toBe("sls+https://example.com/db?db=vault");
expect(stdout.spy).toHaveBeenCalled();
});
it("remote-set updates URI and applies settings when target is active", async () => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://old.example/db",
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-set", ["r1", "sls+https://new.example/db"]), {
...context,
core,
});
expect(result).toBe(true);
expect(settings.remoteConfigurations.r1.uri).toContain("sls+https://new.example/db");
expect(core.services.control.applySettings).toHaveBeenCalledTimes(1);
});
it.each(protocolFixtures)(
"remote-activate projects effective settings for $protocol",
async ({ connectionString, assertProjectedFields }) => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://old.example/?db=old",
isEncrypted: false,
};
settings.remoteConfigurations.r2 = {
id: "r2",
name: "R2",
uri: connectionString,
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-activate", ["r2"]), {
...context,
core,
});
expect(result).toBe(true);
expect(settings.activeConfigurationId).toBe("r2");
assertProjectedFields(settings);
}
);
it.each(protocolFixtures)(
"remote-set projects effective settings for active remote ($protocol)",
async ({ connectionString, assertProjectedFields }) => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://old.example/?db=old",
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-set", ["r1", connectionString]), {
...context,
core,
});
expect(result).toBe(true);
assertProjectedFields(settings);
}
);
it.each(protocolFixtures)(
"remote-rm projects promoted active remote effective settings for $protocol",
async ({ connectionString, assertProjectedFields }) => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteConfigurations.r1 = {
id: "r1",
name: "R1",
uri: "sls+https://old.example/?db=old",
isEncrypted: false,
};
settings.remoteConfigurations.r2 = {
id: "r2",
name: "R2",
uri: connectionString,
isEncrypted: false,
};
settings.activeConfigurationId = "r1";
const result = await runCommand(makeOptions("remote-rm", ["r1"]), {
...context,
core,
});
expect(result).toBe(true);
expect(settings.activeConfigurationId).toBe("r2");
assertProjectedFields(settings);
}
);
it.each([
["couchdb", "sls+https://user:pass@example.com:5984/?db=vault"] as const,
[
"s3",
"sls+s3://ak:sk@example.com/?endpoint=https%3A%2F%2Fs3.example.com&bucket=my-bucket&region=ap-northeast-1",
] as const,
[
"p2p",
"sls+p2p://room-abc?passphrase=pass-123&relays=wss%3A%2F%2Frelay.example&appId=self-hosted-livesync",
] as const,
])("remote command round-trip works for %s", async (_protocol, initialConnStr) => {
const core = createCoreMock();
const addOut = captureStdout();
const addResult = await runCommand(makeOptions("remote-add", ["rt", initialConnStr]), {
...context,
core,
});
expect(addResult).toBe(true);
const remoteId = parseAddedRemoteIdFromLines(addOut.lines());
expect(remoteId).not.toBe("");
const export1Out = captureStdout();
const export1Result = await runCommand(makeOptions("remote-export", [remoteId]), {
...context,
core,
});
expect(export1Result).toBe(true);
const export1Lines = export1Out.lines();
const exported1 = export1Lines.length > 0 ? export1Lines[export1Lines.length - 1] : "";
expect(exported1).toBe(ConnectionStringParser.serialize(ConnectionStringParser.parse(initialConnStr)));
const roundTripInput = ConnectionStringParser.serialize(ConnectionStringParser.parse(exported1));
const setResult = await runCommand(makeOptions("remote-set", [remoteId, roundTripInput]), {
...context,
core,
});
expect(setResult).toBe(true);
const export2Out = captureStdout();
const export2Result = await runCommand(makeOptions("remote-export", [remoteId]), {
...context,
core,
});
expect(export2Result).toBe(true);
const export2Lines = export2Out.lines();
const exported2 = export2Lines.length > 0 ? export2Lines[export2Lines.length - 1] : "";
expect(exported2).toBe(roundTripInput);
});
});

View File

@@ -1,5 +1,6 @@
import { LiveSyncBaseCore } from "../../../LiveSyncBaseCore";
import { ServiceContext } from "@lib/services/base/ServiceBase";
import type { ObsidianLiveSyncSettings } from "@lib/common/types";
export type CLICommand =
| "daemon"
@@ -19,6 +20,12 @@ export type CLICommand =
| "rm"
| "resolve"
| "mirror"
| "remote-add"
| "remote-rm"
| "remote-ls"
| "remote-export"
| "remote-set"
| "remote-activate"
| "init-settings";
export interface CLIOptions {
@@ -29,15 +36,27 @@ export interface CLIOptions {
force?: boolean;
command: CLICommand;
commandArgs: string[];
interval?: number;
}
export interface CLICommandContext {
vaultPath: string;
databasePath: string;
core: LiveSyncBaseCore<ServiceContext, any>;
settingsPath: string;
originalSyncSettings: Pick<
ObsidianLiveSyncSettings,
| "liveSync"
| "syncOnStart"
| "periodicReplication"
| "syncOnSave"
| "syncOnEditorSave"
| "syncOnFileOpen"
| "syncAfterMerge"
>;
}
export const VALID_COMMANDS = new Set([
"daemon",
"sync",
"p2p-peers",
"p2p-sync",
@@ -54,5 +73,11 @@ export const VALID_COMMANDS = new Set([
"rm",
"resolve",
"mirror",
"remote-add",
"remote-rm",
"remote-ls",
"remote-export",
"remote-set",
"remote-activate",
"init-settings",
] as const);

View File

@@ -5,19 +5,19 @@ export function toArrayBuffer(data: Buffer): ArrayBuffer {
return data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength) as ArrayBuffer;
}
export function toVaultRelativePath(inputPath: string, vaultPath: string): string {
export function toDatabaseRelativePath(inputPath: string, databasePath: string): string {
const stripped = inputPath.replace(/^[/\\]+/, "");
if (!path.isAbsolute(inputPath)) {
const normalized = stripped.replace(/\\/g, "/");
const resolved = path.resolve(vaultPath, normalized);
const rel = path.relative(vaultPath, resolved);
const resolved = path.resolve(databasePath, normalized);
const rel = path.relative(databasePath, resolved);
if (rel.startsWith("..") || path.isAbsolute(rel)) {
throw new Error(`Path ${inputPath} is outside of the local database directory`);
}
return rel.replace(/\\/g, "/");
}
const resolved = path.resolve(inputPath);
const rel = path.relative(vaultPath, resolved);
const rel = path.relative(databasePath, resolved);
if (rel.startsWith("..") || path.isAbsolute(rel)) {
throw new Error(`Path ${inputPath} is outside of the local database directory`);
}
@@ -25,15 +25,15 @@ export function toVaultRelativePath(inputPath: string, vaultPath: string): strin
}
export async function readStdinAsUtf8(): Promise<string> {
const chunks: Buffer[] = [];
const chunks = [];
for await (const chunk of process.stdin) {
if (typeof chunk === "string") {
chunks.push(Buffer.from(chunk, "utf-8"));
} else {
chunks.push(chunk);
chunks.push(chunk as Buffer);
}
}
return Buffer.concat(chunks).toString("utf-8");
return Buffer.concat(chunks as Uint8Array[]).toString("utf-8");
}
export async function promptForPassphrase(prompt = "Enter setup URI passphrase: "): Promise<string> {

View File

@@ -1,29 +1,33 @@
import * as path from "path";
import { describe, expect, it } from "vitest";
import { toVaultRelativePath } from "./utils";
import { toDatabaseRelativePath } from "./utils";
describe("toVaultRelativePath", () => {
const vaultPath = path.resolve("/tmp/livesync-vault");
describe("toDatabaseRelativePath", () => {
const databasePath = path.resolve("/tmp/livesync-vault");
it("rejects absolute paths outside vault", () => {
expect(() => toVaultRelativePath("/etc/passwd", vaultPath)).toThrow("outside of the local database directory");
expect(() => toDatabaseRelativePath("/etc/passwd", databasePath)).toThrow(
"outside of the local database directory"
);
});
it("normalizes leading slash for absolute path inside vault", () => {
const absoluteInsideVault = path.join(vaultPath, "notes", "foo.md");
expect(toVaultRelativePath(absoluteInsideVault, vaultPath)).toBe("notes/foo.md");
const absoluteInsideVault = path.join(databasePath, "notes", "foo.md");
expect(toDatabaseRelativePath(absoluteInsideVault, databasePath)).toBe("notes/foo.md");
});
it("normalizes Windows-style separators", () => {
expect(toVaultRelativePath("notes\\daily\\2026-03-12.md", vaultPath)).toBe("notes/daily/2026-03-12.md");
expect(toDatabaseRelativePath("notes\\daily\\2026-03-12.md", databasePath)).toBe("notes/daily/2026-03-12.md");
});
it("returns vault-relative path for another absolute path inside vault", () => {
const absoluteInsideVault = path.join(vaultPath, "docs", "inside.md");
expect(toVaultRelativePath(absoluteInsideVault, vaultPath)).toBe("docs/inside.md");
const absoluteInsideVault = path.join(databasePath, "docs", "inside.md");
expect(toDatabaseRelativePath(absoluteInsideVault, databasePath)).toBe("docs/inside.md");
});
it("rejects relative path traversal that escapes vault", () => {
expect(() => toVaultRelativePath("../escape.md", vaultPath)).toThrow("outside of the local database directory");
expect(() => toDatabaseRelativePath("../escape.md", databasePath)).toThrow(
"outside of the local database directory"
);
});
});

187
src/apps/cli/deploy/install.sh Executable file
View File

@@ -0,0 +1,187 @@
#!/usr/bin/env bash
# install.sh — install livesync-cli as a systemd service
#
# Usage:
# install.sh [--user] [--system] [--vault <path>] [--interval <N>]
#
# Defaults: user install, prompts for vault path if not supplied.
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd -- "$SCRIPT_DIR/../../.." && pwd)"
CLI_DIR="$REPO_ROOT/src/apps/cli"
SERVICE_TEMPLATE="$SCRIPT_DIR/livesync-cli.service"
# ── Argument parsing ────────────────────────────────────────────────────────
INSTALL_MODE="user"
VAULT_PATH=""
INTERVAL=""
FORCE=0
while [[ $# -gt 0 ]]; do
case "$1" in
--user)
INSTALL_MODE="user"
shift
;;
--system)
INSTALL_MODE="system"
shift
;;
--vault)
if [[ -z "${2:-}" ]]; then
echo "Error: --vault requires a path argument" >&2
exit 1
fi
VAULT_PATH="$2"
shift 2
;;
--interval)
if [[ -z "${2:-}" ]]; then
echo "Error: --interval requires a numeric argument" >&2
exit 1
fi
INTERVAL="$2"
if ! [[ "$INTERVAL" =~ ^[1-9][0-9]*$ ]]; then
echo "Error: --interval requires a positive integer, got '$INTERVAL'" >&2
exit 1
fi
shift 2
;;
--force|-f)
FORCE=1
shift
;;
--help|-h)
cat <<EOF
Usage: install.sh [--user|--system] [--vault <path>] [--interval <N>] [--force]
--user Install as a user systemd service (default, ~/.config/systemd/user/)
--system Install as a system systemd service (/etc/systemd/system/)
--vault Path to the vault directory (prompted if omitted)
--interval Poll CouchDB every N seconds instead of using the _changes feed
--force Overwrite existing service unit without prompting
EOF
exit 0
;;
*)
echo "Error: Unknown argument: $1" >&2
exit 1
;;
esac
done
# ── Vault path ──────────────────────────────────────────────────────────────
if [[ -z "$VAULT_PATH" ]]; then
if [ ! -t 0 ]; then
echo "Error: --vault is required in non-interactive mode" >&2
exit 1
fi
printf 'Vault path: '
read -r VAULT_PATH
fi
_orig_vault="$VAULT_PATH"
if ! VAULT_PATH="$(cd -- "$VAULT_PATH" 2>/dev/null && pwd)"; then
echo "Error: vault directory does not exist: $_orig_vault" >&2
exit 1
fi
echo "[INFO] Vault: $VAULT_PATH"
echo "[INFO] Install mode: $INSTALL_MODE"
# ── Build ────────────────────────────────────────────────────────────────────
echo "[INFO] Building CLI from $REPO_ROOT..."
(cd "$REPO_ROOT" && npm install --silent)
(cd "$CLI_DIR" && npm run build)
BUILT_CJS="$CLI_DIR/dist/index.cjs"
if [[ ! -f "$BUILT_CJS" ]]; then
echo "Error: build output not found: $BUILT_CJS" >&2
exit 1
fi
# ── Install binary ───────────────────────────────────────────────────────────
if [[ "$INSTALL_MODE" == "user" ]]; then
BIN_DIR="$HOME/.local/bin"
UNIT_DIR="$HOME/.config/systemd/user"
SYSTEMCTL_FLAGS="--user"
else
BIN_DIR="/usr/local/bin"
UNIT_DIR="/etc/systemd/system"
SYSTEMCTL_FLAGS=""
fi
mkdir -p "$BIN_DIR"
LIVESYNC_BIN="$BIN_DIR/livesync-cli"
LIVESYNC_JS="$BIN_DIR/livesync-cli.js"
# Copy the CJS bundle so the wrapper is self-contained and independent of the
# build directory location.
cp "$BUILT_CJS" "$LIVESYNC_JS"
# Write a bash wrapper that invokes node on the installed bundle.
cat > "$LIVESYNC_BIN" <<WRAPPER
#!/usr/bin/env bash
exec node "$LIVESYNC_JS" "\$@"
WRAPPER
chmod +x "$LIVESYNC_BIN"
echo "[INFO] Installed bundle: $LIVESYNC_JS"
echo "[INFO] Installed binary: $LIVESYNC_BIN"
# ── Write systemd unit ───────────────────────────────────────────────────────
mkdir -p "$UNIT_DIR"
UNIT_PATH="$UNIT_DIR/livesync-cli.service"
EXEC_START="\"$LIVESYNC_BIN\" \"$VAULT_PATH\""
if [[ -n "$INTERVAL" ]]; then
EXEC_START="\"$LIVESYNC_BIN\" \"$VAULT_PATH\" --interval $INTERVAL"
fi
# Check for existing service and offer to overwrite.
if [[ -f "$UNIT_PATH" ]] && [[ "$FORCE" -eq 0 ]]; then
if [ ! -t 0 ]; then
echo "Error: service unit already exists at $UNIT_PATH; use --force to overwrite" >&2
exit 1
fi
printf 'Service unit already exists at %s. Overwrite? [y/N]: ' "$UNIT_PATH"
read -r CONFIRM
case "$CONFIRM" in
[yY]|[yY][eE][sS]) : ;;
*)
echo "[INFO] Aborted. Existing unit left in place."
exit 0
;;
esac
fi
# In awk gsub(), '&' in the replacement means "matched text"; escape any literal '&'
# in path variables before passing them as awk replacement strings.
AWK_BIN="${LIVESYNC_BIN//&/\\&}"
AWK_VAULT="${VAULT_PATH//&/\\&}"
awk -v bin="$AWK_BIN" -v vault="$AWK_VAULT" -v exec_start="ExecStart=$EXEC_START" \
'/^ExecStart=/ { print exec_start; next } {gsub("LIVESYNC_BIN", bin); gsub("LIVESYNC_VAULT_PATH", vault); print}' \
"$SERVICE_TEMPLATE" > "$UNIT_PATH"
echo "[INFO] Installed unit: $UNIT_PATH"
# ── Enable service ───────────────────────────────────────────────────────────
if ! command -v systemctl >/dev/null 2>&1; then
echo "[WARN] systemctl not found — skipping service activation"
echo "[INFO] To enable manually, copy $UNIT_PATH to the correct systemd directory and run:"
echo " systemctl $SYSTEMCTL_FLAGS daemon-reload"
echo " systemctl $SYSTEMCTL_FLAGS enable --now livesync-cli"
exit 0
fi
# shellcheck disable=SC2086
systemctl $SYSTEMCTL_FLAGS daemon-reload
# shellcheck disable=SC2086
systemctl $SYSTEMCTL_FLAGS enable --now livesync-cli
echo ""
echo "[Done] livesync-cli service installed and started."
echo ""
# shellcheck disable=SC2086
systemctl $SYSTEMCTL_FLAGS status livesync-cli --no-pager || true

View File

@@ -0,0 +1,17 @@
[Unit]
Description=Self-hosted LiveSync CLI Daemon
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=LIVESYNC_BIN LIVESYNC_VAULT_PATH
Restart=on-failure
RestartSec=10
TimeoutStartSec=300
StandardOutput=journal
StandardError=journal
LimitNOFILE=65536
[Install]
WantedBy=default.target

View File

@@ -8,7 +8,6 @@ import * as path from "path";
import { NodeServiceContext, NodeServiceHub } from "./services/NodeServiceHub";
import { configureNodeLocalStorage, ensureGlobalNodeLocalStorage } from "./services/NodeLocalStorage";
import { LiveSyncBaseCore } from "../../LiveSyncBaseCore";
import { ModuleReplicatorP2P } from "../../modules/core/ModuleReplicatorP2P";
import { initialiseServiceModulesCLI } from "./serviceModules/CLIServiceModules";
import { DEFAULT_SETTINGS, LOG_LEVEL_VERBOSE, type LOG_LEVEL, type ObsidianLiveSyncSettings } from "@lib/common/types";
import type { InjectableServiceHub } from "@lib/services/implements/injectable/InjectableServiceHub";
@@ -26,6 +25,8 @@ import { VALID_COMMANDS } from "./commands/types";
import type { CLICommand, CLIOptions } from "./commands/types";
import { getPathFromUXFileInfo } from "@lib/common/typeUtils";
import { stripAllPrefixes } from "@lib/string_and_binary/path";
import { IgnoreRules } from "./serviceModules/IgnoreRules";
import { useP2PReplicatorFeature } from "@/lib/src/replication/trystero/useP2PReplicatorFeature";
const SETTINGS_FILE = ".livesync/settings.json";
ensureGlobalNodeLocalStorage();
@@ -36,14 +37,16 @@ function printHelp(): void {
Self-hosted LiveSync CLI
Usage:
livesync-cli [database-path] [options] [command] [command-args]
livesync-cli <database-path> [options] <command> [command-args]
livesync-cli init-settings [path]
Arguments:
database-path Path to the local database directory (required)
database-path Path to the local database directory
Commands:
sync Run one replication cycle and exit
p2p-peers <timeout> Show discovered peers as [peer]<TAB><peer-id><TAB><peer-name>
daemon (default) Run mirror scan then continuously sync CouchDB <-> local filesystem
sync Run one replication cycle and exit
p2p-peers <timeout> Show discovered peers as [peer]\t<peer-id>\t<peer-name>
p2p-sync <peer> <timeout>
Sync with the specified peer-id or peer-name
p2p-host Start P2P host mode and wait until interrupted
@@ -54,11 +57,28 @@ Commands:
put <dst> Read UTF-8 content from stdin and write to local database path <dst>
cat <src> Read file <src> from local database and write to stdout
cat-rev <src> <rev> Read file <src> at specific revision <rev> and write to stdout
ls [prefix] List DB files as path<TAB>size<TAB>mtime<TAB>revision[*]
ls [prefix] List DB files as path\tsize\tmtime\trevision[*]
info <path> Show detailed metadata for a file (ID, revision, conflicts, chunks)
rm <path> Mark a file as deleted in local database
resolve <path> <rev> Resolve conflicts by keeping <rev> and deleting others
mirror [vault-path] Mirror database contents to the local file system (vault-path defaults to database-path)
remote-add <name> <connstr>
Add a remote configuration from a connection string
remote-rm <remote-id> Remove a remote configuration by ID
remote-ls List stored remote configurations
remote-export <remote-id>
Export a remote connection string by ID
remote-set <remote-id> <connstr>
Replace a stored remote connection string by ID
remote-activate <remote-id>
Activate a stored remote configuration by ID
Options:
--interval <N>, -i <N> (daemon only) Poll CouchDB every N seconds instead of using the _changes feed
Examples:
livesync-cli ./my-database Run daemon (LiveSync mode)
livesync-cli ./my-database --interval 30 Run daemon (polling every 30s)
livesync-cli ./my-database sync
livesync-cli ./my-database p2p-peers 5
livesync-cli ./my-database p2p-sync my-peer-name 15
@@ -74,6 +94,12 @@ Examples:
livesync-cli ./my-database info notes/hello.md
livesync-cli ./my-database rm notes/hello.md
livesync-cli ./my-database resolve notes/hello.md 3-abcdef
livesync-cli ./my-database remote-add my-remote "sls+https://user:pass@example.com/db"
livesync-cli ./my-database remote-ls
livesync-cli ./my-database remote-export remote-abc123
livesync-cli ./my-database remote-set remote-abc123 "sls+s3://ak:sk@example.com/?endpoint=https%3A%2F%2Fs3.example.com&bucket=mybucket"
livesync-cli ./my-database remote-activate remote-abc123
livesync-cli ./my-database remote-rm remote-abc123
livesync-cli init-settings ./data.json
livesync-cli ./my-database --verbose
`);
@@ -92,6 +118,7 @@ export function parseArgs(): CLIOptions {
let verbose = false;
let debug = false;
let force = false;
let interval: number | undefined;
let command: CLICommand = "daemon";
const commandArgs: string[] = [];
@@ -108,10 +135,26 @@ export function parseArgs(): CLIOptions {
settingsPath = args[i];
break;
}
case "--interval":
case "-i": {
i++;
if (!args[i]) {
console.error(`Error: Missing value for ${token}`);
process.exit(1);
}
const n = parseInt(args[i], 10);
if (!Number.isInteger(n) || n <= 0) {
console.error(`Error: --interval requires a positive integer, got '${args[i]}'`);
process.exit(1);
}
interval = n;
break;
}
case "--debug":
case "-d":
// debugging automatically enables verbose logging, as it is intended for debugging issues.
debug = true;
// falls through
case "--verbose":
case "-v":
verbose = true;
@@ -161,6 +204,7 @@ export function parseArgs(): CLIOptions {
force,
command,
commandArgs,
interval,
};
}
@@ -194,10 +238,16 @@ async function createDefaultSettingsFile(options: CLIOptions) {
export async function main() {
const options = parseArgs();
if (options.interval && options.command !== "daemon") {
console.error(`Warning: --interval is only used in daemon mode, ignored for '${options.command}'`);
}
const avoidStdoutNoise =
options.command === "cat" ||
options.command === "cat-rev" ||
options.command === "ls" ||
options.command === "remote-add" ||
options.command === "remote-ls" ||
options.command === "remote-export" ||
options.command === "p2p-peers" ||
options.command === "info" ||
options.command === "rm" ||
@@ -220,34 +270,45 @@ export async function main() {
return;
}
// Resolve vault path
const vaultPath = path.resolve(options.databasePath!);
// Check if vault directory exists
// Resolve database path
const databasePath = path.resolve(options.databasePath!);
// Check if database directory exists
try {
const stat = await fs.stat(vaultPath);
const stat = await fs.stat(databasePath);
if (!stat.isDirectory()) {
console.error(`Error: ${vaultPath} is not a directory`);
console.error(`Error: ${databasePath} is not a directory`);
process.exit(1);
}
} catch (error) {
console.error(`Error: Vault directory ${vaultPath} does not exist`);
console.error(`Error: Database directory ${databasePath} does not exist`);
process.exit(1);
}
// Resolve settings path
const settingsPath = options.settingsPath
? path.resolve(options.settingsPath)
: path.join(vaultPath, SETTINGS_FILE);
configureNodeLocalStorage(path.join(vaultPath, ".livesync", "runtime", "local-storage.json"));
: path.join(databasePath, SETTINGS_FILE);
configureNodeLocalStorage(path.join(databasePath, ".livesync", "runtime", "local-storage.json"));
infoLog(`Self-hosted LiveSync CLI`);
infoLog(`Vault: ${vaultPath}`);
infoLog(`Database Path: ${databasePath}`);
infoLog(`Settings: ${settingsPath}`);
infoLog("");
// For daemon and mirror mode, load ignore rules before the core is constructed so that
// chokidar's ignored option is populated when beginWatch() fires during onLoad().
const watchEnabled = options.command === "daemon";
const vaultPath =
options.command === "mirror" && options.commandArgs[0] ? path.resolve(options.commandArgs[0]) : databasePath;
let ignoreRules: IgnoreRules | undefined;
if (options.command === "daemon" || options.command === "mirror") {
ignoreRules = new IgnoreRules(vaultPath);
await ignoreRules.load();
}
// Create service context and hub
const context = new NodeServiceContext(vaultPath);
const serviceHubInstance = new NodeServiceHub<NodeServiceContext>(vaultPath, context);
const context = new NodeServiceContext(databasePath);
const serviceHubInstance = new NodeServiceHub<NodeServiceContext>(databasePath, context);
serviceHubInstance.API.addLog.setHandler((message: string, level: LOG_LEVEL) => {
let levelStr = "";
switch (level) {
@@ -275,11 +336,14 @@ export async function main() {
}
console.error(`${prefix} ${message}`);
});
// Prevent replication result to be processed automatically.
serviceHubInstance.replication.processSynchroniseResult.addHandler(async () => {
console.error(`[Info] Replication result received, but not processed automatically in CLI mode.`);
return await Promise.resolve(true);
}, -100);
// Prevent replication result from being processed automatically in non-daemon commands.
// In daemon mode the default handler must run so changes are applied to the filesystem.
if (options.command !== "daemon") {
serviceHubInstance.replication.processSynchroniseResult.addHandler(async () => {
console.error(`[Info] Replication result received, but not processed automatically in CLI mode.`);
return await Promise.resolve(true);
}, -100);
}
// Setup settings handlers
const settingService = serviceHubInstance.setting;
@@ -321,24 +385,40 @@ export async function main() {
const core = new LiveSyncBaseCore(
serviceHubInstance,
(core: LiveSyncBaseCore<NodeServiceContext, any>, serviceHub: InjectableServiceHub<NodeServiceContext>) => {
return initialiseServiceModulesCLI(vaultPath, core, serviceHub);
return initialiseServiceModulesCLI(vaultPath, core, serviceHub, ignoreRules, watchEnabled);
},
(core) => [
// No modules need to be registered for P2P replication in CLI. Directly using Replicators in p2p.ts
// new ModuleReplicatorP2P(core),
],
(core) => [],
() => [], // No add-ons
(core) => {
// Register P2P replicator feature.
const _replicator = useP2PReplicatorFeature(core);
// Add target filter to prevent internal files are handled
core.services.vault.isTargetFile.addHandler(async (target) => {
const vaultPath = stripAllPrefixes(getPathFromUXFileInfo(target));
const parts = vaultPath.split(path.sep);
const targetPath = stripAllPrefixes(getPathFromUXFileInfo(target));
const parts = targetPath.split(path.sep);
// if some part of the path starts with dot, treat it as internal file and ignore.
if (parts.some((part) => part.startsWith("."))) {
return await Promise.resolve(false);
}
// PouchDB LevelDB database directory lives in the vault directory.
if (parts[0]?.endsWith("-livesync-v2")) {
return await Promise.resolve(false);
}
return await Promise.resolve(true);
}, -1 /* highest priority */);
// Apply user-defined ignore rules for daemon mode (lower priority, runs after dotfile check).
if (ignoreRules) {
const rules = ignoreRules;
core.services.vault.isTargetFile.addHandler(async (target) => {
const targetPath = stripAllPrefixes(getPathFromUXFileInfo(target));
if (rules.shouldIgnore(targetPath)) {
return false;
}
// undefined = pass through to next handler in chain
return undefined;
}, 0);
}
}
);
@@ -359,6 +439,25 @@ export async function main() {
process.on("SIGINT", () => shutdown("SIGINT"));
process.on("SIGTERM", () => shutdown("SIGTERM"));
// Save the settings file before any lifecycle events can mutate and persist them.
// suspendAllSync and other lifecycle hooks clobber sync settings in memory, and
// various code paths persist the clobbered state to disk. We restore on shutdown.
const settingsBackup = await fs.readFile(settingsPath, "utf-8").catch(() => null!);
// Restore settings file on any exit to undo lifecycle mutations.
// Write to a temp path first so a crash mid-write doesn't leave a truncated file.
process.on("exit", () => {
if (settingsBackup) {
const tmpPath = settingsPath + ".tmp";
try {
require("fs").writeFileSync(tmpPath, settingsBackup, "utf-8");
require("fs").renameSync(tmpPath, settingsPath);
} catch (err) {
console.error("[Settings] Failed to restore settings on exit:", err);
}
}
});
// Start the core
try {
infoLog(`[Starting] Initializing LiveSync...`);
@@ -368,6 +467,18 @@ export async function main() {
console.error(`[Error] Failed to initialize LiveSync`);
process.exit(1);
}
// Capture sync settings before suspendAllSync() clobbers them.
// Used by daemon mode to restore the correct sync behaviour after the mirror scan.
const settingsBeforeSuspend = core.services.setting.currentSettings();
const originalSyncSettings = {
liveSync: settingsBeforeSuspend.liveSync,
syncOnStart: settingsBeforeSuspend.syncOnStart,
periodicReplication: settingsBeforeSuspend.periodicReplication,
syncOnSave: settingsBeforeSuspend.syncOnSave,
syncOnEditorSave: settingsBeforeSuspend.syncOnEditorSave,
syncOnFileOpen: settingsBeforeSuspend.syncOnFileOpen,
syncAfterMerge: settingsBeforeSuspend.syncAfterMerge,
};
await core.services.setting.suspendAllSync();
await core.services.control.onReady();
@@ -393,7 +504,7 @@ export async function main() {
infoLog("");
}
const result = await runCommand(options, { vaultPath, core, settingsPath });
const result = await runCommand(options, { databasePath, core, settingsPath, originalSyncSettings });
if (!result) {
console.error(`[Error] Command '${options.command}' failed`);
process.exitCode = 1;
@@ -401,7 +512,7 @@ export async function main() {
infoLog(`[Done] Command '${options.command}' completed`);
}
if (options.command === "daemon") {
if (options.command === "daemon" && result) {
// Keep the process running
await new Promise(() => {});
} else {

View File

@@ -17,7 +17,7 @@ describe("CLI parseArgs", () => {
});
it("exits 1 when --settings has no value", () => {
process.argv = ["node", "livesync-cli", "./vault", "--settings"];
process.argv = ["node", "livesync-cli", "./databasePath", "--settings"];
const exitMock = mockProcessExit();
const stderr = vi.spyOn(console, "error").mockImplementation(() => {});
@@ -37,7 +37,7 @@ describe("CLI parseArgs", () => {
});
it("exits 1 for unknown command after database-path", () => {
process.argv = ["node", "livesync-cli", "./vault", "unknown-cmd"];
process.argv = ["node", "livesync-cli", "./databasePath", "unknown-cmd"];
const exitMock = mockProcessExit();
const stderr = vi.spyOn(console, "error").mockImplementation(() => {});
@@ -56,33 +56,146 @@ describe("CLI parseArgs", () => {
expect(stdout).toHaveBeenCalled();
const combined = stdout.mock.calls.flat().join("\n");
expect(combined).toContain("Usage:");
expect(combined).toContain("livesync-cli [database-path]");
expect(combined).toContain("livesync-cli <database-path> [options] <command> [command-args]");
});
it("parses p2p-peers command and timeout", () => {
process.argv = ["node", "livesync-cli", "./vault", "p2p-peers", "5"];
process.argv = ["node", "livesync-cli", "./databasePath", "p2p-peers", "5"];
const parsed = parseArgs();
expect(parsed.databasePath).toBe("./vault");
expect(parsed.databasePath).toBe("./databasePath");
expect(parsed.command).toBe("p2p-peers");
expect(parsed.commandArgs).toEqual(["5"]);
});
it("parses p2p-sync command with peer and timeout", () => {
process.argv = ["node", "livesync-cli", "./vault", "p2p-sync", "peer-1", "12"];
process.argv = ["node", "livesync-cli", "./databasePath", "p2p-sync", "peer-1", "12"];
const parsed = parseArgs();
expect(parsed.databasePath).toBe("./vault");
expect(parsed.databasePath).toBe("./databasePath");
expect(parsed.command).toBe("p2p-sync");
expect(parsed.commandArgs).toEqual(["peer-1", "12"]);
});
it("parses p2p-host command", () => {
process.argv = ["node", "livesync-cli", "./vault", "p2p-host"];
process.argv = ["node", "livesync-cli", "./databasePath", "p2p-host"];
const parsed = parseArgs();
expect(parsed.databasePath).toBe("./vault");
expect(parsed.databasePath).toBe("./databasePath");
expect(parsed.command).toBe("p2p-host");
expect(parsed.commandArgs).toEqual([]);
});
it("parses remote-add command", () => {
process.argv = [
"node",
"livesync-cli",
"./databasePath",
"remote-add",
"my-remote",
"sls+https://user:pass@example.com/db",
];
const parsed = parseArgs();
expect(parsed.databasePath).toBe("./databasePath");
expect(parsed.command).toBe("remote-add");
expect(parsed.commandArgs).toEqual(["my-remote", "sls+https://user:pass@example.com/db"]);
});
it("parses remote-activate command", () => {
process.argv = ["node", "livesync-cli", "./databasePath", "remote-activate", "remote-abc"];
const parsed = parseArgs();
expect(parsed.databasePath).toBe("./databasePath");
expect(parsed.command).toBe("remote-activate");
expect(parsed.commandArgs).toEqual(["remote-abc"]);
});
it("parses remote-export command", () => {
process.argv = ["node", "livesync-cli", "./databasePath", "remote-export", "remote-abc"];
const parsed = parseArgs();
expect(parsed.databasePath).toBe("./databasePath");
expect(parsed.command).toBe("remote-export");
expect(parsed.commandArgs).toEqual(["remote-abc"]);
});
it("parses remote-set command", () => {
process.argv = [
"node",
"livesync-cli",
"./databasePath",
"remote-set",
"remote-abc",
"sls+p2p://room-1?passphrase=abc",
];
const parsed = parseArgs();
expect(parsed.databasePath).toBe("./databasePath");
expect(parsed.command).toBe("remote-set");
expect(parsed.commandArgs).toEqual(["remote-abc", "sls+p2p://room-1?passphrase=abc"]);
});
it("parses --interval flag with valid integer", () => {
process.argv = ["node", "livesync-cli", "./vault", "--interval", "30"];
const parsed = parseArgs();
expect(parsed.command).toBe("daemon");
expect(parsed.interval).toBe(30);
});
it("parses -i shorthand for --interval", () => {
process.argv = ["node", "livesync-cli", "./vault", "-i", "10"];
const parsed = parseArgs();
expect(parsed.interval).toBe(10);
});
it("exits 1 when --interval has no value", () => {
process.argv = ["node", "livesync-cli", "./vault", "--interval"];
const exitMock = mockProcessExit();
vi.spyOn(console, "error").mockImplementation(() => {});
expect(() => parseArgs()).toThrowError("__EXIT__:1");
expect(exitMock).toHaveBeenCalledWith(1);
});
it("exits 1 when --interval is not a positive integer", () => {
process.argv = ["node", "livesync-cli", "./vault", "--interval", "0"];
const exitMock = mockProcessExit();
vi.spyOn(console, "error").mockImplementation(() => {});
expect(() => parseArgs()).toThrowError("__EXIT__:1");
expect(exitMock).toHaveBeenCalledWith(1);
});
it("exits 1 when --interval is negative", () => {
process.argv = ["node", "livesync-cli", "./vault", "--interval", "-5"];
const exitMock = mockProcessExit();
vi.spyOn(console, "error").mockImplementation(() => {});
expect(() => parseArgs()).toThrowError("__EXIT__:1");
});
it("exits 1 when --interval is not numeric", () => {
process.argv = ["node", "livesync-cli", "./vault", "--interval", "abc"];
const exitMock = mockProcessExit();
vi.spyOn(console, "error").mockImplementation(() => {});
expect(() => parseArgs()).toThrowError("__EXIT__:1");
});
it("parses explicit daemon command", () => {
process.argv = ["node", "livesync-cli", "./vault", "daemon"];
const parsed = parseArgs();
expect(parsed.command).toBe("daemon");
expect(parsed.databasePath).toBe("./vault");
});
it("defaults to daemon when no command specified", () => {
process.argv = ["node", "livesync-cli", "./vault"];
const parsed = parseArgs();
expect(parsed.command).toBe("daemon");
});
it("parses explicit daemon command with --interval", () => {
process.argv = ["node", "livesync-cli", "./vault", "daemon", "--interval", "30"];
const parsed = parseArgs();
expect(parsed.command).toBe("daemon");
expect(parsed.interval).toBe(30);
});
});

View File

@@ -11,8 +11,11 @@ import type {
} from "@lib/managers/adapters";
import type { FileEventItemSentinel } from "@lib/managers/StorageEventManager";
import type { NodeFile, NodeFolder } from "../adapters/NodeTypes";
import type { Stats } from "fs";
import * as fs from "fs/promises";
import * as path from "path";
import { watch as chokidarWatch, type FSWatcher } from "chokidar";
import type { IgnoreRules } from "../serviceModules/IgnoreRules";
/**
* CLI-specific type guard adapter
@@ -56,22 +59,11 @@ class CLIPersistenceAdapter implements IStorageEventPersistenceAdapter {
}
/**
* CLI-specific status adapter (console logging)
* CLI-specific status adapter (no-op — daemon uses journald for status)
*/
class CLIStatusAdapter implements IStorageEventStatusAdapter {
private lastUpdate = 0;
private updateInterval = 5000; // Update every 5 seconds
updateStatus(status: { batched: number; processing: number; totalQueued: number }): void {
const now = Date.now();
if (now - this.lastUpdate > this.updateInterval) {
if (status.totalQueued > 0 || status.processing > 0) {
// console.log(
// `[StorageEventManager] Batched: ${status.batched}, Processing: ${status.processing}, Total Queued: ${status.totalQueued}`
// );
}
this.lastUpdate = now;
}
updateStatus(_status: { batched: number; processing: number; totalQueued: number }): void {
// intentional no-op
}
}
@@ -100,15 +92,101 @@ class CLIConverterAdapter implements IStorageEventConverterAdapter<NodeFile> {
}
/**
* CLI-specific watch adapter (optional file watching with chokidar)
* CLI-specific watch adapter using chokidar for real-time filesystem monitoring.
*/
class CLIWatchAdapter implements IStorageEventWatchAdapter {
constructor(private basePath: string) {}
private _watcher: FSWatcher | undefined;
constructor(
private basePath: string,
private ignoreRules?: IgnoreRules,
private watchEnabled: boolean = false
) {}
private _toNodeFile(filePath: string, stats: Stats | undefined): NodeFile {
return {
path: path.relative(this.basePath, filePath).replace(/\\/g, "/") as FilePath,
stat: {
ctime: stats?.ctimeMs ?? Date.now(),
mtime: stats?.mtimeMs ?? Date.now(),
size: stats?.size ?? 0,
type: "file",
},
};
}
private _toNodeFolder(dirPath: string): NodeFolder {
return {
path: path.relative(this.basePath, dirPath).replace(/\\/g, "/") as FilePath,
isFolder: true,
};
}
async beginWatch(handlers: IStorageEventWatchHandlers): Promise<void> {
// File watching is not activated in the CLI.
// Because the CLI is designed for push/pull operations, not real-time sync.
// console.error("[CLIWatchAdapter] File watching is not enabled in CLI version");
if (!this.watchEnabled) return;
const baseIgnored: Array<RegExp | string | ((p: string) => boolean)> = [
/(^|[/\\])\./,
/(^|[/\\])[^/\\]*-livesync-v2([/\\]|$)/,
];
// Bind rules to a local const before the closure — chokidar v4 requires a
// MatchFunction, not glob strings, for custom patterns.
const rules = this.ignoreRules;
const ignored = rules
? [...baseIgnored, (p: string) => rules.shouldIgnore(path.relative(this.basePath, p))]
: baseIgnored;
const watcher = chokidarWatch(this.basePath, {
ignored,
ignoreInitial: true,
persistent: true,
awaitWriteFinish: {
stabilityThreshold: 500,
pollInterval: 100,
},
});
watcher.on("add", (filePath, stats) => {
const nodeFile = this._toNodeFile(filePath, stats);
handlers.onCreate(nodeFile);
});
watcher.on("change", (filePath, stats) => {
const nodeFile = this._toNodeFile(filePath, stats);
handlers.onChange(nodeFile);
});
watcher.on("unlink", (filePath) => {
const nodeFile = this._toNodeFile(filePath, undefined);
handlers.onDelete(nodeFile);
});
watcher.on("addDir", (dirPath) => {
const nodeFolder = this._toNodeFolder(dirPath);
handlers.onCreate(nodeFolder);
});
watcher.on("unlinkDir", (dirPath) => {
const nodeFolder = this._toNodeFolder(dirPath);
handlers.onDelete(nodeFolder);
});
watcher.on("error", (err) => {
console.error("[CLIWatchAdapter] Fatal watcher error — file watching stopped:", err);
console.error("[CLIWatchAdapter] Exiting for systemd restart.");
void watcher.close();
this._watcher = undefined;
// Use exit(1) rather than SIGTERM so systemd Restart=on-failure engages.
process.exit(1);
});
await new Promise<void>((resolve) => watcher.once("ready", resolve));
this._watcher = watcher;
}
close(): Promise<void> {
if (this._watcher) {
return this._watcher.close();
}
return Promise.resolve();
}
}
@@ -123,11 +201,15 @@ export class CLIStorageEventManagerAdapter implements IStorageEventManagerAdapte
readonly status: CLIStatusAdapter;
readonly converter: CLIConverterAdapter;
constructor(basePath: string) {
constructor(basePath: string, ignoreRules?: IgnoreRules, watchEnabled: boolean = false) {
this.typeGuard = new CLITypeGuardAdapter();
this.persistence = new CLIPersistenceAdapter(basePath);
this.watch = new CLIWatchAdapter(basePath);
this.watch = new CLIWatchAdapter(basePath, ignoreRules, watchEnabled);
this.status = new CLIStatusAdapter();
this.converter = new CLIConverterAdapter();
}
close(): Promise<void> {
return this.watch.close();
}
}

View File

@@ -0,0 +1,123 @@
import { describe, expect, it, vi, beforeEach } from "vitest";
import type { IStorageEventWatchHandlers } from "@lib/managers/adapters";
import type { NodeFile } from "../adapters/NodeTypes";
// ── chokidar mock ──────────────────────────────────────────────────────────────
// Must be hoisted before imports that pull in chokidar.
const mockWatcher = {
on: vi.fn().mockReturnThis(),
once: vi.fn((event: string, cb: () => void) => {
if (event === "ready") cb();
return mockWatcher;
}),
close: vi.fn(() => Promise.resolve()),
};
vi.mock("chokidar", () => ({
watch: vi.fn(() => mockWatcher),
}));
import * as chokidar from "chokidar";
import { CLIStorageEventManagerAdapter } from "./CLIStorageEventManagerAdapter";
// ── helpers ───────────────────────────────────────────────────────────────────
function makeHandlers(): IStorageEventWatchHandlers {
return {
onCreate: vi.fn(),
onChange: vi.fn(),
onDelete: vi.fn(),
onRename: vi.fn(),
} as any;
}
// ── tests ─────────────────────────────────────────────────────────────────────
describe("CLIStorageEventManagerAdapter", () => {
beforeEach(() => {
vi.clearAllMocks();
// Restore the default once() behaviour (ready fires synchronously).
mockWatcher.once.mockImplementation((event: string, cb: () => void) => {
if (event === "ready") cb();
return mockWatcher;
});
});
it("beginWatch is no-op when watchEnabled=false", async () => {
const adapter = new CLIStorageEventManagerAdapter("/base", undefined, false);
const handlers = makeHandlers();
await adapter.watch.beginWatch(handlers);
expect(chokidar.watch).not.toHaveBeenCalled();
});
it("beginWatch calls chokidar.watch when watchEnabled=true", async () => {
const adapter = new CLIStorageEventManagerAdapter("/base", undefined, true);
const handlers = makeHandlers();
await adapter.watch.beginWatch(handlers);
expect(chokidar.watch).toHaveBeenCalledTimes(1);
expect(chokidar.watch).toHaveBeenCalledWith("/base", expect.objectContaining({ ignoreInitial: true }));
});
it("add event produces NodeFile with correct relative path via onCreate", async () => {
const basePath = "/vault/base";
const adapter = new CLIStorageEventManagerAdapter(basePath, undefined, true);
const handlers = makeHandlers();
await adapter.watch.beginWatch(handlers);
// Find the callback registered for the "add" event.
const addCall = mockWatcher.on.mock.calls.find(([event]) => event === "add");
expect(addCall).toBeDefined();
const addCallback = addCall![1] as (filePath: string, stats: any) => void;
const fakeStats = { ctimeMs: 1000, mtimeMs: 2000, size: 42 };
addCallback(`${basePath}/subdir/note.md`, fakeStats);
expect(handlers.onCreate).toHaveBeenCalledTimes(1);
const created = (handlers.onCreate as ReturnType<typeof vi.fn>).mock.calls[0][0] as NodeFile;
expect(created.path).toBe("subdir/note.md");
expect(created.stat?.size).toBe(42);
});
it("close() calls watcher.close()", async () => {
const adapter = new CLIStorageEventManagerAdapter("/base", undefined, true);
const handlers = makeHandlers();
await adapter.watch.beginWatch(handlers);
await adapter.close();
expect(mockWatcher.close).toHaveBeenCalledTimes(1);
});
it("close() is safe when no watcher was started", async () => {
const adapter = new CLIStorageEventManagerAdapter("/base", undefined, false);
// Should not throw.
await expect(adapter.close()).resolves.toBeUndefined();
expect(mockWatcher.close).not.toHaveBeenCalled();
});
it("error event triggers process.exit(1)", async () => {
const adapter = new CLIStorageEventManagerAdapter("/base", undefined, true);
const handlers = makeHandlers();
await adapter.watch.beginWatch(handlers);
const processExitSpy = vi.spyOn(process, "exit").mockImplementation((() => {}) as any);
const errorCall = mockWatcher.on.mock.calls.find(([event]) => event === "error");
expect(errorCall).toBeDefined();
const errorCallback = errorCall![1] as (err: Error) => void;
errorCallback(new Error("disk failure"));
expect(processExitSpy).toHaveBeenCalledWith(1);
processExitSpy.mockRestore();
});
});

View File

@@ -2,6 +2,7 @@ import { StorageEventManagerBase, type StorageEventManagerBaseDependencies } fro
import { CLIStorageEventManagerAdapter } from "./CLIStorageEventManagerAdapter";
import type { IMinimumLiveSyncCommands, LiveSyncBaseCore } from "../../../LiveSyncBaseCore";
import type { ServiceContext } from "@lib/services/base/ServiceBase";
import type { IgnoreRules } from "../serviceModules/IgnoreRules";
// import type { IMinimumLiveSyncCommands } from "@lib/services/base/IService";
export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEventManagerAdapter> {
@@ -10,9 +11,11 @@ export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEv
constructor(
basePath: string,
core: LiveSyncBaseCore<ServiceContext, IMinimumLiveSyncCommands>,
dependencies: StorageEventManagerBaseDependencies
dependencies: StorageEventManagerBaseDependencies,
ignoreRules?: IgnoreRules,
watchEnabled?: boolean
) {
const adapter = new CLIStorageEventManagerAdapter(basePath);
const adapter = new CLIStorageEventManagerAdapter(basePath, ignoreRules, watchEnabled);
super(adapter, dependencies);
this.core = core;
}
@@ -25,4 +28,11 @@ export class StorageEventManagerCLI extends StorageEventManagerBase<CLIStorageEv
// No-op in CLI version
// Internal file handling is not needed
}
/**
* Close the file watcher. Call this during graceful shutdown.
*/
close(): Promise<void> {
return this.adapter.close();
}
}

View File

@@ -6,6 +6,7 @@
"type": "module",
"scripts": {
"dev": "vite",
"prebuild": "node scripts/check-submodule.mjs",
"build": "vite build",
"preview": "vite preview",
"cli": "node dist/index.cjs",

View File

@@ -4,6 +4,7 @@
"version": "0.0.0",
"description": "Runtime dependencies for Self-hosted LiveSync CLI Docker image",
"dependencies": {
"chokidar": "^4.0.0",
"commander": "^14.0.3",
"werift": "^0.22.9",
"pouchdb-adapter-http": "^9.0.0",

View File

@@ -0,0 +1,36 @@
import fs from "node:fs";
import path from "node:path";
import process from "node:process";
const cliDir = process.cwd();
const repoRoot = path.resolve(cliDir, "../../..");
const requiredFiles = [
path.join(repoRoot, "src/lib/src/common/types.ts"),
];
const missingFiles = requiredFiles.filter((filePath) => !fs.existsSync(filePath));
if (missingFiles.length === 0) {
process.exit(0);
}
console.error("[CLI Build Error] Required shared sources were not found.");
console.error("This repository uses Git submodules, and the CLI depends on src/lib.");
console.error("");
console.error("Missing file(s):");
for (const filePath of missingFiles) {
console.error(` - ${path.relative(repoRoot, filePath)}`);
}
console.error("");
console.error("Initialize submodules, then retry the CLI build:");
console.error(" git submodule update --init --recursive");
console.error("");
console.error("For a fresh clone, prefer:");
console.error(" git clone --recurse-submodules <repository-url>");
console.error("");
console.error("Then run:");
console.error(" npm install");
console.error(" cd src/apps/cli");
console.error(" npm run build");
process.exit(1);

View File

@@ -9,6 +9,7 @@ import { ServiceFileAccessCLI } from "./ServiceFileAccessImpl";
import { ServiceDatabaseFileAccessCLI } from "./DatabaseFileAccess";
import { StorageEventManagerCLI } from "../managers/StorageEventManagerCLI";
import type { ServiceModules } from "@lib/interfaces/ServiceModule";
import type { IgnoreRules } from "./IgnoreRules";
/**
* Initialize service modules for CLI version
@@ -22,7 +23,9 @@ import type { ServiceModules } from "@lib/interfaces/ServiceModule";
export function initialiseServiceModulesCLI(
basePath: string,
core: LiveSyncBaseCore<ServiceContext, any>,
services: InjectableServiceHub<ServiceContext>
services: InjectableServiceHub<ServiceContext>,
ignoreRules?: IgnoreRules,
watchEnabled: boolean = false
): ServiceModules {
const storageAccessManager = new StorageAccessManager();
@@ -36,12 +39,24 @@ export function initialiseServiceModulesCLI(
});
// CLI-specific storage event manager
const storageEventManager = new StorageEventManagerCLI(basePath, core, {
fileProcessing: services.fileProcessing,
setting: services.setting,
vaultService: services.vault,
storageAccessManager: storageAccessManager,
APIService: services.API,
const storageEventManager = new StorageEventManagerCLI(
basePath,
core,
{
fileProcessing: services.fileProcessing,
setting: services.setting,
vaultService: services.vault,
storageAccessManager: storageAccessManager,
APIService: services.API,
},
ignoreRules,
watchEnabled
);
// Close the file watcher during graceful shutdown so the process can exit cleanly.
services.appLifecycle.onUnload.addHandler(async () => {
await storageEventManager.close();
return true;
});
// Storage access using CLI file system adapter

View File

@@ -0,0 +1,131 @@
import * as fs from "fs/promises";
import * as path from "path";
import { minimatch } from "minimatch";
/**
* Loads and evaluates ignore rules from `.livesync/ignore` inside the vault.
*
* File format:
* - Lines starting with `#` are comments.
* - Blank lines are ignored.
* - `import: .gitignore` (exactly) — merges patterns from the vault's `.gitignore`.
* - All other lines are minimatch glob patterns relative to the vault root.
*
* Negation patterns (lines starting with `!`) are not supported. Loading a
* ruleset containing them throws an error — use separate include/exclude files
* instead.
*
* Missing files (`.livesync/ignore` or `.gitignore`) are silently skipped.
*/
export class IgnoreRules {
private patterns: string[] = [];
constructor(private vaultPath: string) {}
/**
* Reads `.livesync/ignore` (and optionally `.gitignore`) and populates the
* pattern list. Safe to call multiple times — each call replaces the
* previous state. Does not throw if files are absent.
*
* @throws if any pattern line begins with `!` (negation is unsupported).
*/
async load(): Promise<void> {
this.patterns = [];
const ignorePath = path.join(this.vaultPath, ".livesync", "ignore");
let rawLines: string[];
try {
const content = await fs.readFile(ignorePath, "utf-8");
rawLines = content.split(/\r?\n/);
} catch {
// File absent or unreadable — treat as empty ruleset.
return;
}
for (const line of rawLines) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith("#")) {
continue;
}
// NOTE: Only the exact string "import: .gitignore" is recognised.
// Any future generalisation of this directive must validate that
// the resolved path stays within the vault directory.
if (trimmed === "import: .gitignore") {
await this._importGitignore();
continue;
}
if (trimmed.startsWith("import:")) {
console.error(
`[IgnoreRules] Warning: unrecognised directive '${trimmed}' — only 'import: .gitignore' is supported`
);
continue;
}
this._addPattern(trimmed);
}
if (this.patterns.length > 0) {
console.error(`[IgnoreRules] Loaded ${this.patterns.length} ignore patterns`);
}
}
// Normalises a single gitignore-style pattern:
// - Patterns ending with `/` (directory patterns like `build/`) are
// converted to `build/**` so they match all files inside that directory.
// - Patterns without a `/` are prefixed with `**/` to give them matchBase
// semantics (e.g. `*.tmp` → `**/*.tmp`), matching the basename in any
// subdirectory as gitignore does.
// - Patterns that already contain a `/` (but don't end with one) are
// path-specific and used as-is.
private _normalisePattern(pattern: string): string {
if (pattern.endsWith("/")) {
return "**/" + pattern + "**";
} else if (!pattern.includes("/")) {
return "**/" + pattern;
}
return pattern;
}
private async _importGitignore(): Promise<void> {
const gitignorePath = path.join(this.vaultPath, ".gitignore");
let content: string;
try {
content = await fs.readFile(gitignorePath, "utf-8");
} catch {
return;
}
this._parseLines(content);
}
private _parseLines(content: string): void {
for (const line of content.split(/\r?\n/)) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith("#")) continue;
this._addPattern(trimmed);
}
}
private _addPattern(raw: string): void {
if (raw.startsWith("!")) {
throw new Error(
`[IgnoreRules] Negation pattern '${raw}' is not supported. ` +
`Remove it from .livesync/ignore or use a separate include/exclude file.`
);
}
this.patterns.push(this._normalisePattern(raw));
}
/**
* Returns `true` if the given vault-relative path matches any loaded
* ignore pattern.
*
* @param relativePath - Path relative to the vault root, using forward
* slashes or the OS separator.
*/
shouldIgnore(relativePath: string): boolean {
if (this.patterns.length === 0) {
return false;
}
// Normalise to forward slashes for minimatch.
const normalised = relativePath.replace(/\\/g, "/");
return this.patterns.some((p) => minimatch(normalised, p, { dot: true }));
}
}

View File

@@ -0,0 +1,169 @@
import * as fs from "node:fs/promises";
import * as os from "node:os";
import * as path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { IgnoreRules } from "./IgnoreRules";
describe("IgnoreRules", () => {
const tempDirs: string[] = [];
async function createVault(): Promise<string> {
const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "livesync-ignorerules-"));
tempDirs.push(tempDir);
return tempDir;
}
async function writeIgnoreFile(vaultPath: string, content: string): Promise<void> {
const ignoreDir = path.join(vaultPath, ".livesync");
await fs.mkdir(ignoreDir, { recursive: true });
await fs.writeFile(path.join(ignoreDir, "ignore"), content, "utf-8");
}
afterEach(async () => {
await Promise.all(tempDirs.splice(0).map((dir) => fs.rm(dir, { recursive: true, force: true })));
});
describe("pattern normalisation", () => {
it("adds **/ prefix to basename patterns (no slash)", async () => {
const vaultPath = await createVault();
await writeIgnoreFile(vaultPath, "*.tmp\n");
const rules = new IgnoreRules(vaultPath);
await rules.load();
expect(rules.shouldIgnore("notes/scratch.tmp")).toBe(true);
expect(rules.shouldIgnore("scratch.tmp")).toBe(true);
expect(rules.shouldIgnore("deep/nested/file.tmp")).toBe(true);
});
it("appends ** to directory patterns ending with / and prepends **/", async () => {
const vaultPath = await createVault();
await writeIgnoreFile(vaultPath, "build/\n");
const rules = new IgnoreRules(vaultPath);
await rules.load();
expect(rules.shouldIgnore("build/output.js")).toBe(true);
expect(rules.shouldIgnore("build/nested/file.js")).toBe(true);
expect(rules.shouldIgnore("subproject/build/output.js")).toBe(true);
});
it("leaves patterns containing / as-is", async () => {
const vaultPath = await createVault();
await writeIgnoreFile(vaultPath, "docs/private.md\n");
const rules = new IgnoreRules(vaultPath);
await rules.load();
expect(rules.shouldIgnore("docs/private.md")).toBe(true);
expect(rules.shouldIgnore("other/docs/private.md")).toBe(false);
});
});
describe("shouldIgnore", () => {
it("matches **/*.tmp against notes/scratch.tmp", async () => {
const vaultPath = await createVault();
await writeIgnoreFile(vaultPath, "*.tmp\n");
const rules = new IgnoreRules(vaultPath);
await rules.load();
expect(rules.shouldIgnore("notes/scratch.tmp")).toBe(true);
});
it("does not match notes/readme.md against **/*.tmp", async () => {
const vaultPath = await createVault();
await writeIgnoreFile(vaultPath, "*.tmp\n");
const rules = new IgnoreRules(vaultPath);
await rules.load();
expect(rules.shouldIgnore("notes/readme.md")).toBe(false);
});
it("returns false when no patterns are loaded", async () => {
const vaultPath = await createVault();
const rules = new IgnoreRules(vaultPath);
// No load() call — patterns are empty
expect(rules.shouldIgnore("anything.md")).toBe(false);
});
});
describe("negation patterns", () => {
it("throws when a negation pattern is encountered", async () => {
const vaultPath = await createVault();
await writeIgnoreFile(vaultPath, "*.tmp\n!important.tmp\n");
const rules = new IgnoreRules(vaultPath);
await expect(rules.load()).rejects.toThrow(/Negation pattern/);
});
it("throws when a .gitignore imported via directive contains negation", async () => {
const vaultPath = await createVault();
await writeIgnoreFile(vaultPath, "import: .gitignore\n");
await fs.writeFile(path.join(vaultPath, ".gitignore"), "*.log\n!keep.log\n", "utf-8");
const rules = new IgnoreRules(vaultPath);
await expect(rules.load()).rejects.toThrow(/Negation pattern/);
});
});
describe("unrecognised import: directives", () => {
it("warns and skips unrecognised import: forms (does not add as literal pattern)", async () => {
const vaultPath = await createVault();
// Typo: "import:.gitignore" instead of "import: .gitignore"
await writeIgnoreFile(vaultPath, "*.tmp\nimport:.gitignore\n");
const rules = new IgnoreRules(vaultPath);
await rules.load();
// *.tmp still loaded; import:.gitignore is skipped (not treated as a literal pattern)
expect(rules.shouldIgnore("scratch.tmp")).toBe(true);
expect(rules.shouldIgnore("import:.gitignore")).toBe(false);
});
});
describe("load() with missing file", () => {
it("returns without error when .livesync/ignore is absent", async () => {
const vaultPath = await createVault();
// No ignore file created
const rules = new IgnoreRules(vaultPath);
await expect(rules.load()).resolves.toBeUndefined();
expect(rules.shouldIgnore("anything.md")).toBe(false);
});
});
describe("load() with comments and blank lines", () => {
it("skips # comment lines and blank lines", async () => {
const vaultPath = await createVault();
await writeIgnoreFile(vaultPath, "# This is a comment\n\n \n*.tmp\n# another comment\nbuild/\n");
const rules = new IgnoreRules(vaultPath);
await rules.load();
expect(rules.shouldIgnore("scratch.tmp")).toBe(true);
expect(rules.shouldIgnore("build/output.js")).toBe(true);
expect(rules.shouldIgnore("readme.md")).toBe(false);
});
});
describe("import: .gitignore directive", () => {
it("reads and normalises patterns from .gitignore", async () => {
const vaultPath = await createVault();
await writeIgnoreFile(vaultPath, "import: .gitignore\n");
await fs.writeFile(path.join(vaultPath, ".gitignore"), "*.log\nnode_modules/\n", "utf-8");
const rules = new IgnoreRules(vaultPath);
await rules.load();
expect(rules.shouldIgnore("app.log")).toBe(true);
expect(rules.shouldIgnore("node_modules/package.json")).toBe(true);
expect(rules.shouldIgnore("src/node_modules/package.json")).toBe(true);
expect(rules.shouldIgnore("src/index.ts")).toBe(false);
});
it("merges .gitignore patterns with other patterns", async () => {
const vaultPath = await createVault();
await writeIgnoreFile(vaultPath, "*.tmp\nimport: .gitignore\n");
await fs.writeFile(path.join(vaultPath, ".gitignore"), "*.log\n", "utf-8");
const rules = new IgnoreRules(vaultPath);
await rules.load();
expect(rules.shouldIgnore("scratch.tmp")).toBe(true);
expect(rules.shouldIgnore("error.log")).toBe(true);
});
});
describe("import: .gitignore with missing .gitignore", () => {
it("does not throw when .gitignore is absent", async () => {
const vaultPath = await createVault();
await writeIgnoreFile(vaultPath, "*.tmp\nimport: .gitignore\n");
// No .gitignore created
const rules = new IgnoreRules(vaultPath);
await expect(rules.load()).resolves.toBeUndefined();
// The *.tmp pattern from the ignore file still works
expect(rules.shouldIgnore("scratch.tmp")).toBe(true);
});
});
});

View File

@@ -27,10 +27,10 @@ import { DatabaseService } from "@lib/services/base/DatabaseService";
import type { ObsidianLiveSyncSettings } from "@/lib/src/common/types";
export class NodeServiceContext extends ServiceContext {
vaultPath: string;
constructor(vaultPath: string) {
databasePath: string;
constructor(databasePath: string) {
super();
this.vaultPath = vaultPath;
this.databasePath = databasePath;
}
}
@@ -64,7 +64,7 @@ class NodeDatabaseService<T extends NodeServiceContext> extends DatabaseService<
): { name: string; options: PouchDB.Configuration.DatabaseConfiguration } {
const optionPass = {
...options,
prefix: this.context.vaultPath + nodePath.sep,
prefix: this.context.databasePath + nodePath.sep,
};
const passSettings = { ...settings, useIndexedDBAdapter: false };
return super.modifyDatabaseOptions(passSettings, name, optionPass);

View File

@@ -0,0 +1,49 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
CLI_DIR="$(cd -- "$SCRIPT_DIR/.." && pwd)"
cd "$CLI_DIR"
source "$SCRIPT_DIR/test-helpers.sh"
display_test_info "Test for Issue #860: Empty output from ls and mirror"
RUN_BUILD="${RUN_BUILD:-1}"
cli_test_init_cli_cmd
WORK_DIR="$(mktemp -d "${TMPDIR:-/tmp}/livesync-repro-860.XXXXXX")"
trap 'rm -rf "$WORK_DIR"' EXIT
SETTINGS_FILE="$WORK_DIR/data.json"
VAULT_DIR="$WORK_DIR/vault"
mkdir -p "$VAULT_DIR"
if [[ "$RUN_BUILD" == "1" ]]; then
echo "[INFO] building CLI..."
npm run build
fi
echo "[INFO] generating settings -> $SETTINGS_FILE"
cli_test_init_settings_file "$SETTINGS_FILE"
# 1. Test 'ls' on empty database
echo "[INFO] Testing 'ls' on empty database..."
LS_OUTPUT=$(run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" ls)
if [[ -z "$LS_OUTPUT" ]]; then
echo "[REPRODUCED] 'ls' returned empty output for empty database."
else
echo "[INFO] 'ls' output: $LS_OUTPUT"
fi
# 2. Test 'mirror' on empty vault
echo "[INFO] Testing 'mirror' on empty vault..."
MIRROR_OUTPUT=$(run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" mirror 2>&1)
if [[ "$MIRROR_OUTPUT" == *"[Command] mirror"* ]] && [[ ! "$MIRROR_OUTPUT" == *"[Mirror]"* ]]; then
# Note: currently it prints [Command] mirror to stderr.
# Let's see if it prints anything else.
echo "[REPRODUCED] 'mirror' produced no functional logs (only command header)."
else
echo "[INFO] 'mirror' output: $MIRROR_OUTPUT"
fi
echo "[DONE] finished repro-860 test"

View File

@@ -0,0 +1,166 @@
#!/usr/bin/env bash
# Test: daemon-related ignore rules behaviour
#
# Tests that are runnable without a long-running daemon process are exercised
# here using the `mirror` command, which calls the same `isTargetFile` handler
# stack that the daemon uses.
#
# Covered cases:
# 1. .livesync/ignore with *.tmp pattern → ignored file is NOT synced to DB
# 2. .livesync/ignore missing → no error, normal sync continues
# 3. import: .gitignore directive → patterns from .gitignore are merged
#
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
CLI_DIR="$(cd -- "$SCRIPT_DIR/.." && pwd)"
cd "$CLI_DIR"
source "$SCRIPT_DIR/test-helpers.sh"
display_test_info
RUN_BUILD="${RUN_BUILD:-1}"
cli_test_init_cli_cmd
WORK_DIR="$(mktemp -d "${TMPDIR:-/tmp}/livesync-cli-daemon-test.XXXXXX")"
trap 'rm -rf "$WORK_DIR"' EXIT
SETTINGS_FILE="$WORK_DIR/data.json"
VAULT_DIR="$WORK_DIR/vault"
mkdir -p "$VAULT_DIR/notes"
if [[ "$RUN_BUILD" == "1" ]]; then
echo "[INFO] building CLI..."
npm run build
fi
echo "[INFO] generating settings -> $SETTINGS_FILE"
cli_test_init_settings_file "$SETTINGS_FILE"
cli_test_mark_settings_configured "$SETTINGS_FILE"
PASS=0
FAIL=0
assert_pass() { echo "[PASS] $1"; PASS=$((PASS + 1)); }
assert_fail() { echo "[FAIL] $1" >&2; FAIL=$((FAIL + 1)); }
# ─────────────────────────────────────────────────────────────────────────────
# Case 1: .livesync/ignore with *.tmp → matched file should NOT appear in DB
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "=== Case 1: .livesync/ignore *.tmp → ignored file not synced to DB ==="
mkdir -p "$VAULT_DIR/.livesync"
printf '*.tmp\n' > "$VAULT_DIR/.livesync/ignore"
# Also write a normal file so we can confirm mirror ran at all.
printf 'normal content\n' > "$VAULT_DIR/notes/normal.md"
# Write the file that should be ignored.
printf 'tmp content\n' > "$VAULT_DIR/notes/scratch.tmp"
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" mirror
# The normal file should be in the DB.
RESULT_NORMAL="$WORK_DIR/case1-normal.txt"
if run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" pull notes/normal.md "$RESULT_NORMAL" 2>/dev/null; then
if cmp -s "$VAULT_DIR/notes/normal.md" "$RESULT_NORMAL"; then
assert_pass "normal.md was synced to DB"
else
assert_fail "normal.md content mismatch after mirror"
fi
else
assert_fail "normal.md was not found in DB after mirror"
fi
# The .tmp file should NOT be in the DB.
DB_LIST="$WORK_DIR/case1-ls.txt"
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" ls > "$DB_LIST"
if grep -q "scratch.tmp" "$DB_LIST"; then
assert_fail "scratch.tmp (ignored) was unexpectedly synced to DB"
echo "--- DB listing ---" >&2; cat "$DB_LIST" >&2
else
assert_pass "scratch.tmp (*.tmp pattern) was NOT synced to DB"
fi
# ─────────────────────────────────────────────────────────────────────────────
# Case 2: .livesync/ignore absent → no error, normal sync continues
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "=== Case 2: .livesync/ignore absent → no error, sync continues ==="
VAULT_DIR2="$WORK_DIR/vault2"
mkdir -p "$VAULT_DIR2/notes"
SETTINGS_FILE2="$WORK_DIR/data2.json"
cli_test_init_settings_file "$SETTINGS_FILE2"
cli_test_mark_settings_configured "$SETTINGS_FILE2"
# No .livesync directory at all.
printf 'hello\n' > "$VAULT_DIR2/notes/hello.md"
# mirror should succeed without error.
set +e
MIRROR_OUTPUT="$WORK_DIR/case2-mirror.txt"
run_cli "$VAULT_DIR2" --settings "$SETTINGS_FILE2" mirror >"$MIRROR_OUTPUT" 2>&1
MIRROR_EXIT=$?
set -e
if [[ "$MIRROR_EXIT" -ne 0 ]]; then
assert_fail "mirror exited non-zero ($MIRROR_EXIT) when .livesync/ignore is absent"
cat "$MIRROR_OUTPUT" >&2
else
assert_pass "mirror succeeded when .livesync/ignore is absent"
fi
# The normal file should have been synced.
RESULT_HELLO="$WORK_DIR/case2-hello.txt"
if run_cli "$VAULT_DIR2" --settings "$SETTINGS_FILE2" pull notes/hello.md "$RESULT_HELLO" 2>/dev/null; then
assert_pass "file synced normally when .livesync/ignore is absent"
else
assert_fail "file was not synced when .livesync/ignore is absent"
fi
# ─────────────────────────────────────────────────────────────────────────────
# Case 3: import: .gitignore merges patterns
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "=== Case 3: import: .gitignore directive merges patterns ==="
VAULT_DIR3="$WORK_DIR/vault3"
mkdir -p "$VAULT_DIR3/notes"
SETTINGS_FILE3="$WORK_DIR/data3.json"
cli_test_init_settings_file "$SETTINGS_FILE3"
cli_test_mark_settings_configured "$SETTINGS_FILE3"
mkdir -p "$VAULT_DIR3/.livesync"
printf 'import: .gitignore\n' > "$VAULT_DIR3/.livesync/ignore"
printf '# gitignore comment\n*.log\nbuild/\n' > "$VAULT_DIR3/.gitignore"
printf 'regular note\n' > "$VAULT_DIR3/notes/regular.md"
printf 'log content\n' > "$VAULT_DIR3/notes/debug.log"
run_cli "$VAULT_DIR3" --settings "$SETTINGS_FILE3" mirror
DB_LIST3="$WORK_DIR/case3-ls.txt"
run_cli "$VAULT_DIR3" --settings "$SETTINGS_FILE3" ls > "$DB_LIST3"
if grep -q "debug.log" "$DB_LIST3"; then
assert_fail "debug.log (ignored via .gitignore import) was unexpectedly synced to DB"
echo "--- DB listing ---" >&2; cat "$DB_LIST3" >&2
else
assert_pass "debug.log (*.log from imported .gitignore) was NOT synced to DB"
fi
# regular.md should still be present.
if grep -q "regular.md" "$DB_LIST3"; then
assert_pass "regular.md was synced normally alongside .gitignore import rules"
else
assert_fail "regular.md was NOT synced — .gitignore import may have been too broad"
fi
# ─────────────────────────────────────────────────────────────────────────────
# Summary
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "Results: PASS=$PASS FAIL=$FAIL"
if [[ "$FAIL" -gt 0 ]]; then
exit 1
fi

83
src/apps/cli/test/test-mirror-linux.sh Normal file → Executable file
View File

@@ -28,7 +28,9 @@ trap 'rm -rf "$WORK_DIR"' EXIT
SETTINGS_FILE="$WORK_DIR/data.json"
VAULT_DIR="$WORK_DIR/vault"
DB_DIR="$WORK_DIR/db"
mkdir -p "$VAULT_DIR/test"
mkdir -p "$DB_DIR"
if [[ "$RUN_BUILD" == "1" ]]; then
echo "[INFO] building CLI..."
@@ -41,6 +43,20 @@ cli_test_init_settings_file "$SETTINGS_FILE"
# isConfigured=true is required for mirror (canProceedScan checks this)
cli_test_mark_settings_configured "$SETTINGS_FILE"
# Preparation: Sync settings and files logic
DB_SETTINGS="$DB_DIR/settings.json"
cp "$SETTINGS_FILE" "$DB_SETTINGS"
# Helper for standard run (Separated paths)
run_mirror_test() {
run_cli "$DB_DIR" --settings "$DB_SETTINGS" mirror "$VAULT_DIR"
}
# Helper for compatibility run (Same path)
run_mirror_compat() {
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" mirror
}
PASS=0
FAIL=0
@@ -78,19 +94,27 @@ portable_touch_timestamp() {
# Case 1: File exists only in storage → should be synced into DB after mirror
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "=== Case 1: storage-only → DB ==="
echo "=== Case 1: storage-only → DB (Separated Paths) ==="
printf 'storage-only content\n' > "$VAULT_DIR/test/storage-only.md"
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" mirror
echo "[DEBUG] DB_DIR: $DB_DIR"
echo "[DEBUG] VAULT_DIR: $VAULT_DIR"
run_mirror_test
RESULT_FILE="$WORK_DIR/case1-cat.txt"
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" pull test/storage-only.md "$RESULT_FILE"
# Try 'ls' first to see what's in the DB
echo "--- DB contents ---"
run_cli "$DB_DIR" --settings "$DB_SETTINGS" ls
echo "-------------------"
run_cli "$DB_DIR" --settings "$DB_SETTINGS" pull test/storage-only.md "$RESULT_FILE"
if cmp -s "$VAULT_DIR/test/storage-only.md" "$RESULT_FILE"; then
assert_pass "storage-only file was synced into DB"
assert_pass "storage-only file was synced into DB using separated paths"
else
assert_fail "storage-only file NOT synced into DB"
assert_fail "storage-only file NOT synced into DB with separated paths"
echo "--- storage ---" >&2; cat "$VAULT_DIR/test/storage-only.md" >&2
echo "--- cat ---" >&2; cat "$RESULT_FILE" >&2
fi
@@ -99,9 +123,9 @@ fi
# Case 2: File exists only in DB → should be restored to storage after mirror
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "=== Case 2: DB-only → storage ==="
echo "=== Case 2: DB-only → storage (Separated Paths) ==="
printf 'db-only content\n' | run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" put test/db-only.md
printf 'db-only content\n' | run_cli "$DB_DIR" --settings "$DB_SETTINGS" put test/db-only.md
if [[ -f "$VAULT_DIR/test/db-only.md" ]]; then
assert_fail "db-only.md unexpectedly exists in storage before mirror"
@@ -109,7 +133,7 @@ else
echo "[INFO] confirmed: test/db-only.md not in storage before mirror"
fi
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" mirror
run_mirror_test
if [[ -f "$VAULT_DIR/test/db-only.md" ]]; then
STORAGE_CONTENT="$(cat "$VAULT_DIR/test/db-only.md")"
@@ -119,19 +143,19 @@ if [[ -f "$VAULT_DIR/test/db-only.md" ]]; then
assert_fail "DB-only file restored but content mismatch (got: '${STORAGE_CONTENT}')"
fi
else
assert_fail "DB-only file was NOT restored to storage"
assert_fail "DB-only file NOT restored to storage after mirror"
fi
# ─────────────────────────────────────────────────────────────────────────────
# Case 3: File deleted in DB → should NOT be created in storage
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "=== Case 3: DB-deleted → storage untouched ==="
echo "=== Case 3: DB-deleted → storage untouched (Separated Paths) ==="
printf 'to-be-deleted\n' | run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" put test/deleted.md
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" rm test/deleted.md
printf 'to-be-deleted\n' | run_cli "$DB_DIR" --settings "$DB_SETTINGS" put test/deleted.md
run_cli "$DB_DIR" --settings "$DB_SETTINGS" rm test/deleted.md
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" mirror
run_mirror_test
if [[ ! -f "$VAULT_DIR/test/deleted.md" ]]; then
assert_pass "deleted DB entry was not restored to storage"
@@ -143,19 +167,19 @@ fi
# Case 4: Both exist, storage is newer → DB should be updated
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "=== Case 4: storage newer → DB updated ==="
echo "=== Case 4: storage newer → DB updated (Separated Paths) ==="
# Seed DB with old content (mtime ≈ now)
printf 'old content\n' | run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" put test/sync-storage-newer.md
printf 'old content\n' | run_cli "$DB_DIR" --settings "$DB_SETTINGS" put test/sync-storage-newer.md
# Write new content to storage with a timestamp 1 hour in the future
printf 'new content\n' > "$VAULT_DIR/test/sync-storage-newer.md"
touch -t "$(portable_touch_timestamp '+1 hour')" "$VAULT_DIR/test/sync-storage-newer.md"
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" mirror
run_mirror_test
DB_RESULT_FILE="$WORK_DIR/case4-pull.txt"
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" pull test/sync-storage-newer.md "$DB_RESULT_FILE"
run_cli "$DB_DIR" --settings "$DB_SETTINGS" pull test/sync-storage-newer.md "$DB_RESULT_FILE"
if cmp -s "$VAULT_DIR/test/sync-storage-newer.md" "$DB_RESULT_FILE"; then
assert_pass "DB updated to match newer storage file"
else
@@ -168,16 +192,16 @@ fi
# Case 5: Both exist, DB is newer → storage should be updated
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "=== Case 5: DB newer → storage updated ==="
echo "=== Case 5: DB newer → storage updated (Separated Paths) ==="
# Write old content to storage with a timestamp 1 hour in the past
printf 'old storage content\n' > "$VAULT_DIR/test/sync-db-newer.md"
touch -t "$(portable_touch_timestamp '-1 hour')" "$VAULT_DIR/test/sync-db-newer.md"
# Write new content to DB only (mtime ≈ now, newer than the storage file)
printf 'new db content\n' | run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" put test/sync-db-newer.md
printf 'new db content\n' | run_cli "$DB_DIR" --settings "$DB_SETTINGS" put test/sync-db-newer.md
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" mirror
run_mirror_test
STORAGE_CONTENT="$(cat "$VAULT_DIR/test/sync-db-newer.md")"
if [[ "$STORAGE_CONTENT" == "new db content" ]]; then
@@ -186,6 +210,25 @@ else
assert_fail "storage NOT updated to match newer DB entry (got: '${STORAGE_CONTENT}')"
fi
# ─────────────────────────────────────────────────────────────────────────────
# Case 6: Compatibility test - omitted vault-path
# ─────────────────────────────────────────────────────────────────────────────
echo ""
echo "=== Case 6: omitted vault-path (Compatibility Mode) ==="
# We use VAULT_DIR as the "main" database path for this part.
printf 'compat-content\n' > "$VAULT_DIR/compat.md"
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" mirror
# In compat mode, it should find it in the DB at root
CAT_RESULT="$WORK_DIR/compat-cat.txt"
run_cli "$VAULT_DIR" --settings "$SETTINGS_FILE" pull compat.md "$CAT_RESULT"
if [[ "$(cat "$CAT_RESULT")" == "compat-content" ]]; then
assert_pass "Compatibility mode works (omitted vault-path)"
else
assert_fail "Compatibility mode failed to sync file into DB"
fi
# ─────────────────────────────────────────────────────────────────────────────
# Summary
# ─────────────────────────────────────────────────────────────────────────────

View File

@@ -0,0 +1,9 @@
hostname=http://127.0.0.1:5989/
dbname=livesync-test-db-ci
username=admin
password=testpassword
minioEndpoint=http://127.0.0.1:9000
accessKey=minioadmin
secretKey=minioadmin
bucketName=livesync-test-bucket-ci
LIVESYNC_TEST_TEE=1

View File

@@ -0,0 +1,150 @@
# Writing CLI Tests on Deno
This guide explains how to add or update tests under `src/apps/cli/testdeno/`.
Note that new tests should be added to the Deno suite rather than the existing bash suite due to the cross-platform execution and TypeScript benefits.
## Scope
The Deno suite is designed for cross-platform execution, with a strong focus on Windows compatibility while keeping behaviour equivalent to existing bash tests.
## Principles
- Keep one scenario per file when practical.
- Reuse helpers from `helpers/` rather than duplicating process, Docker, or settings logic.
- Prefer deterministic data over random inputs unless randomness is explicitly required.
- Ensure every test can clean up automatically.
- Keep assertions actionable with clear failure messages.
## Directory structure
```
src/apps/cli/testdeno/
helpers/
backgroundCli.ts
cli.ts
docker.ts
env.ts
p2p.ts
settings.ts
temp.ts
test-*.ts
deno.json
```
## Test file naming
- Use `test-<feature>.ts`.
- Use names aligned with existing bash tests when porting, for example:
- `test-sync-locked-remote.ts`
- `test-p2p-sync.ts`
## Core helper usage
### Temporary workspace
Use `TempDir` and `await using` so cleanup is automatic:
```ts
await using workDir = await TempDir.create("livesync-cli-my-test");
```
### CLI execution
- `runCli(...)`: returns code and combined output.
- `runCliOrFail(...)`: throws on non-zero exit.
- `runCliWithInputOrFail(input, ...)`: for `put` and stdin-driven commands.
### Settings
- `initSettingsFile(...)`: creates a baseline settings file.
- `applyCouchdbSettings(...)`: applies CouchDB fields.
- `applyRemoteSyncSettings(...)`: applies remote and encryption fields.
- `applyP2pSettings(...)`: applies P2P fields.
- `applyP2pTestTweaks(...)`: enables P2P-only test profile.
### Docker services
- `startCouchdb(...)`, `stopCouchdb()`
- `startP2pRelay()`, `stopP2pRelay()`
### P2P discovery
- `discoverPeer(...)`
- `maybeStartLocalRelay(...)`
- `stopLocalRelayIfStarted(...)`
### Background host process
Use `startCliInBackground(...)` for long-running host mode such as `p2p-host`.
## Recommended test structure
1. Arrange
2. Act
3. Assert
4. Cleanup in `finally`
Example skeleton:
```ts
Deno.test("feature: behaviour", async () => {
await using workDir = await TempDir.create("example");
// Arrange
try {
// Act
// Assert
} finally {
// Optional explicit cleanup
}
});
```
## Reliability guidelines
- Use explicit waits only when needed for eventual consistency.
- Re-run sync operations where the protocol is eventually consistent.
- For network-sensitive commands, use `LIVESYNC_CLI_RETRY` during debugging.
- Keep Docker container reuse disabled by default unless debugging.
## Environment variables
Common variables:
- `LIVESYNC_DOCKER_MODE`
- `LIVESYNC_DOCKER_COMMAND`
- `LIVESYNC_TEST_TEE`
- `LIVESYNC_DOCKER_TEE`
- `LIVESYNC_CLI_DEBUG`
- `LIVESYNC_CLI_VERBOSE`
- `LIVESYNC_CLI_RETRY`
- `LIVESYNC_DEBUG_KEEP_DOCKER`
P2P variables:
- `RELAY`
- `ROOM_ID`
- `PASSPHRASE`
- `APP_ID`
- `PEERS_TIMEOUT`
- `SYNC_TIMEOUT`
- `USE_INTERNAL_RELAY`
## Adding a new test task
1. Add the test file under `src/apps/cli/testdeno/`.
2. Add a task in `src/apps/cli/testdeno/deno.json`.
3. Update `src/apps/cli/testdeno/test_dev_deno.md`.
4. Run the new task locally.
## Validation checklist
- The test passes on a clean workspace.
- The test does not leave persistent artefacts unless explicitly requested.
- Failure messages identify both expected and actual behaviour.
- The corresponding task is documented.
## Out of scope for this suite
- One-off reproduction scripts that are not intended as stable regression tests.

View File

@@ -0,0 +1,22 @@
{
"tasks": {
"test": "deno test --env-file=.test.env -A --no-check test-*.ts",
"test:local": "deno test --env-file=.test.env -A --no-check test-setup-put-cat.ts test-mirror.ts",
"test:push-pull": "deno test --env-file=.test.env -A --no-check test-push-pull.ts",
"test:setup-put-cat": "deno test --env-file=.test.env -A --no-check test-setup-put-cat.ts",
"test:mirror": "deno test --env-file=.test.env -A --no-check test-mirror.ts",
"test:sync-two-local": "deno test --env-file=.test.env -A --no-check test-sync-two-local-databases.ts",
"test:sync-locked-remote": "deno test --env-file=.test.env -A --no-check test-sync-locked-remote.ts",
"test:p2p-host": "deno test --env-file=.test.env -A --no-check test-p2p-host.ts",
"test:p2p-peers": "deno test --env-file=.test.env -A --no-check test-p2p-peers-local-relay.ts",
"test:p2p-sync": "deno test --env-file=.test.env -A --no-check test-p2p-sync.ts",
"test:p2p-three-nodes": "deno test --env-file=.test.env -A --no-check test-p2p-three-nodes-conflict.ts",
"test:p2p-upload-download": "deno test --env-file=.test.env -A --no-check test-p2p-upload-download-repro.ts",
"test:e2e-couchdb": "deno test --env-file=.test.env -A --no-check test-e2e-two-vaults-couchdb.ts",
"test:e2e-matrix": "deno test --env-file=.test.env -A --no-check test-e2e-two-vaults-matrix.ts"
},
"imports": {
"@std/assert": "jsr:@std/assert@^1.0.13",
"@std/path": "jsr:@std/path@^1.0.9"
}
}

31
src/apps/cli/testdeno/deno.lock generated Normal file
View File

@@ -0,0 +1,31 @@
{
"version": "5",
"specifiers": {
"jsr:@std/assert@^1.0.13": "1.0.19",
"jsr:@std/internal@^1.0.12": "1.0.12",
"jsr:@std/path@^1.0.9": "1.1.4"
},
"jsr": {
"@std/assert@1.0.19": {
"integrity": "eaada96ee120cb980bc47e040f82814d786fe8162ecc53c91d8df60b8755991e",
"dependencies": [
"jsr:@std/internal"
]
},
"@std/internal@1.0.12": {
"integrity": "972a634fd5bc34b242024402972cd5143eac68d8dffaca5eaa4dba30ce17b027"
},
"@std/path@1.1.4": {
"integrity": "1d2d43f39efb1b42f0b1882a25486647cb851481862dc7313390b2bb044314b5",
"dependencies": [
"jsr:@std/internal"
]
}
},
"workspace": {
"dependencies": [
"jsr:@std/assert@^1.0.13",
"jsr:@std/path@^1.0.9"
]
}
}

View File

@@ -0,0 +1,112 @@
import { CLI_DIR } from "./cli.ts";
import { join } from "@std/path";
const CLI_DIST = join(CLI_DIR, "dist", "index.cjs");
const VERBOSE_ENABLED = Deno.env.get("LIVESYNC_CLI_VERBOSE") === "1";
const DEBUG_ENABLED = Deno.env.get("LIVESYNC_CLI_DEBUG") === "1";
function decorateArgs(args: string[]): string[] {
return DEBUG_ENABLED ? ["-d", ...args] : VERBOSE_ENABLED ? ["-v", ...args] : args;
}
async function pump(
stream: ReadableStream<Uint8Array>,
sink: (text: string) => void,
teeTarget: WritableStream<Uint8Array> | null
): Promise<void> {
const reader = stream.getReader();
const writer = teeTarget?.getWriter();
const dec = new TextDecoder();
try {
while (true) {
const { done, value } = await reader.read();
if (done) break;
if (!value) continue;
sink(dec.decode(value, { stream: true }));
if (writer) {
await writer.write(value);
}
}
} finally {
if (writer) writer.releaseLock();
reader.releaseLock();
}
}
export class BackgroundCliProcess {
#stdout = "";
#stderr = "";
#stdoutDone: Promise<void>;
#stderrDone: Promise<void>;
constructor(
readonly child: Deno.ChildProcess,
readonly args: string[]
) {
this.#stdoutDone = pump(
child.stdout,
(text) => {
this.#stdout += text;
},
null
);
this.#stderrDone = pump(
child.stderr,
(text) => {
this.#stderr += text;
},
null
);
}
get stdout(): string {
return this.#stdout;
}
get stderr(): string {
return this.#stderr;
}
get combined(): string {
return this.#stdout + this.#stderr;
}
async waitUntilContains(needle: string, timeoutMs = 15000): Promise<void> {
const started = Date.now();
while (Date.now() - started < timeoutMs) {
if (this.combined.includes(needle)) return;
const status = await Promise.race([
this.child.status.then((s) => ({ type: "status" as const, status: s })),
new Promise<{ type: "tick" }>((resolve) => setTimeout(() => resolve({ type: "tick" }), 100)),
]);
if (status.type === "status") {
throw new Error(
`Background CLI exited before '${needle}' appeared (code ${status.status.code})\n${this.combined}`
);
}
}
throw new Error(`Timed out waiting for '${needle}'\n${this.combined}`);
}
async stop(): Promise<number> {
try {
this.child.kill("SIGTERM");
} catch {
// ignore already-exited processes
}
const status = await this.child.status;
await Promise.all([this.#stdoutDone, this.#stderrDone]);
return status.code;
}
}
export function startCliInBackground(...args: string[]): BackgroundCliProcess {
const child = new Deno.Command("node", {
args: [CLI_DIST, ...decorateArgs(args)],
cwd: CLI_DIR,
stdin: "null",
stdout: "piped",
stderr: "piped",
}).spawn();
return new BackgroundCliProcess(child, args);
}

View File

@@ -0,0 +1,231 @@
import { join } from "@std/path";
// ---------------------------------------------------------------------------
// Path resolution
// ---------------------------------------------------------------------------
// This file lives at: src/apps/cli/testdeno/helpers/cli.ts
// CLI root (src/apps/cli/) is two levels up.
// import.meta.dirname is available in Deno 1.40+ as an OS-native path string.
export const CLI_DIR: string = join(import.meta.dirname!, "..", "..");
const CLI_DIST = join(CLI_DIR, "dist", "index.cjs");
// ---------------------------------------------------------------------------
// Result type
// ---------------------------------------------------------------------------
export interface CliResult {
stdout: string;
stderr: string;
/** stdout + stderr concatenated — useful for assertion messages. */
combined: string;
code: number;
}
const TEE_ENABLED = Deno.env.get("LIVESYNC_TEST_TEE") === "1";
const VERBOSE_ENABLED = Deno.env.get("LIVESYNC_CLI_VERBOSE") === "1";
const DEBUG_ENABLED = Deno.env.get("LIVESYNC_CLI_DEBUG") === "1";
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function concatChunks(chunks: Uint8Array[]): Uint8Array {
const total = chunks.reduce((n, c) => n + c.length, 0);
const out = new Uint8Array(total);
let offset = 0;
for (const c of chunks) {
out.set(c, offset);
offset += c.length;
}
return out;
}
async function collectStream(
stream: ReadableStream<Uint8Array>,
teeTarget: WritableStream<Uint8Array> | null
): Promise<Uint8Array> {
const reader = stream.getReader();
const chunks: Uint8Array[] = [];
const writer = teeTarget?.getWriter();
try {
while (true) {
const { done, value } = await reader.read();
if (done) break;
if (value) {
chunks.push(value);
if (writer) {
await writer.write(value);
}
}
}
} finally {
if (writer) {
writer.releaseLock();
}
reader.releaseLock();
}
return concatChunks(chunks);
}
async function runNodeCommand(args: string[], stdinData?: Uint8Array): Promise<CliResult> {
const cliArgs = DEBUG_ENABLED ? ["-d", ...args] : VERBOSE_ENABLED ? ["-v", ...args] : args;
const child = new Deno.Command("node", {
args: [CLI_DIST, ...cliArgs],
cwd: CLI_DIR,
stdin: stdinData ? "piped" : "null",
stdout: "piped",
stderr: "piped",
}).spawn();
const stdoutPromise = collectStream(child.stdout, TEE_ENABLED ? Deno.stdout.writable : null);
const stderrPromise = collectStream(child.stderr, TEE_ENABLED ? Deno.stderr.writable : null);
if (stdinData) {
const w = child.stdin.getWriter();
await w.write(stdinData);
await w.close();
}
const [status, stdout, stderr] = await Promise.all([child.status, stdoutPromise, stderrPromise]);
const dec = new TextDecoder();
const out = dec.decode(stdout);
const err = dec.decode(stderr);
return { stdout: out, stderr: err, combined: out + err, code: status.code };
}
function isTransientNetworkError(message: string): boolean {
const m = message.toLowerCase();
return (
m.includes("fetch failed") ||
m.includes("econnreset") ||
m.includes("econnrefused") ||
m.includes("und_err_socket") ||
m.includes("other side closed")
);
}
// ---------------------------------------------------------------------------
// Core runners
// ---------------------------------------------------------------------------
/**
* Run the CLI (node dist/index.cjs) with the supplied arguments.
* Pass the vault / DB path as the first argument, exactly as the bash helpers
* do. Does NOT throw on non-zero exit — check `.code` yourself.
*/
export async function runCli(...args: string[]): Promise<CliResult> {
const retries = Number(Deno.env.get("LIVESYNC_CLI_RETRY") ?? "0");
for (let attempt = 0; ; attempt++) {
const result = await runNodeCommand(args);
if (result.code === 0) return result;
if (attempt >= retries || !isTransientNetworkError(result.combined)) {
return result;
}
const waitMs = 400 * (attempt + 1);
console.warn(`[WARN] transient CLI failure, retrying (${attempt + 1}/${retries}) in ${waitMs}ms`);
await sleep(waitMs);
}
}
/**
* Run the CLI and throw if it exits non-zero. Returns stdout.
*/
export async function runCliOrFail(...args: string[]): Promise<string> {
const r = await runCli(...args);
if (r.code !== 0) {
throw new Error(`CLI exited with code ${r.code}\nstdout: ${r.stdout}\nstderr: ${r.stderr}`);
}
return r.stdout;
}
/**
* Run the CLI with data piped to stdin (equivalent to `echo … | run_cli …`
* or `cat file | run_cli …`).
*/
export async function runCliWithInput(input: string | Uint8Array, ...args: string[]): Promise<CliResult> {
const data = typeof input === "string" ? new TextEncoder().encode(input) : input;
const retries = Number(Deno.env.get("LIVESYNC_CLI_RETRY") ?? "0");
for (let attempt = 0; ; attempt++) {
const result = await runNodeCommand(args, data);
if (result.code === 0) return result;
if (attempt >= retries || !isTransientNetworkError(result.combined)) {
return result;
}
const waitMs = 400 * (attempt + 1);
console.warn(`[WARN] transient CLI(stdin) failure, retrying (${attempt + 1}/${retries}) in ${waitMs}ms`);
await sleep(waitMs);
}
}
/**
* runCliWithInput — throws on non-zero exit, returns stdout.
*/
export async function runCliWithInputOrFail(input: string | Uint8Array, ...args: string[]): Promise<string> {
const r = await runCliWithInput(input, ...args);
if (r.code !== 0) {
throw new Error(`CLI (with stdin) exited with code ${r.code}\nstdout: ${r.stdout}\nstderr: ${r.stderr}`);
}
return r.stdout;
}
// ---------------------------------------------------------------------------
// Output helpers
// ---------------------------------------------------------------------------
/** Strip the CLIWatchAdapter banner line that `cat` emits. */
export function sanitiseCatStdout(raw: string): string {
return raw
.split("\n")
.filter((l) => l !== "[CLIWatchAdapter] File watching is not enabled in CLI version")
.join("\n");
}
// ---------------------------------------------------------------------------
// Assertions (parity with test-helpers.sh)
// ---------------------------------------------------------------------------
export function assertContains(haystack: string, needle: string, message: string): void {
if (!haystack.includes(needle)) {
throw new Error(`[FAIL] ${message}\nExpected to find: ${JSON.stringify(needle)}\nActual output:\n${haystack}`);
}
}
export function assertNotContains(haystack: string, needle: string, message: string): void {
if (haystack.includes(needle)) {
throw new Error(`[FAIL] ${message}\nDid NOT expect: ${JSON.stringify(needle)}\nActual output:\n${haystack}`);
}
}
export async function assertFilesEqual(expectedPath: string, actualPath: string, message: string): Promise<void> {
const [expected, actual] = await Promise.all([Deno.readFile(expectedPath), Deno.readFile(actualPath)]);
if (expected.length !== actual.length || expected.some((b, i) => b !== actual[i])) {
const hex = async (d: Uint8Array<ArrayBuffer>) => {
const h = await crypto.subtle.digest("SHA-256", d);
return [...new Uint8Array(h)].map((b) => b.toString(16).padStart(2, "0")).join("");
};
throw new Error(
`[FAIL] ${message}\nexpected SHA-256: ${await hex(expected)}\nactual SHA-256: ${await hex(actual)}`
);
}
}
// ---------------------------------------------------------------------------
// JSON helpers
// ---------------------------------------------------------------------------
export async function readJsonFile<T = Record<string, unknown>>(filePath: string): Promise<T> {
return JSON.parse(await Deno.readTextFile(filePath)) as T;
}
export function jsonStringField(jsonText: string, field: string): string {
const data = JSON.parse(jsonText) as Record<string, unknown>;
const value = data[field];
return typeof value === "string" ? value : "";
}
export function jsonFieldIsNa(data: Record<string, unknown>, field: string): boolean {
return data[field] === "N/A";
}

View File

@@ -0,0 +1,530 @@
/**
* Docker service management for tests.
*
* CouchDB start/stop/init is implemented directly using `docker` CLI commands
* and the Fetch API, so it works on any platform where Docker (Desktop) is
* available — including Windows — without needing bash.
*/
type DockerInvoker = {
bin: string;
prefix: string[];
label: string;
};
let dockerInvokerPromise: Promise<DockerInvoker> | null = null;
const DOCKER_TEE = Deno.env.get("LIVESYNC_DOCKER_TEE") === "1" || Deno.env.get("LIVESYNC_TEST_TEE") === "1";
// ---------------------------------------------------------------------------
// Low-level docker wrapper
// ---------------------------------------------------------------------------
function parseCommand(command: string): { bin: string; prefix: string[] } {
const parts = command.trim().split(/\s+/).filter(Boolean);
if (parts.length === 0) {
throw new Error("LIVESYNC_DOCKER_COMMAND is empty");
}
return { bin: parts[0], prefix: parts.slice(1) };
}
async function runCommand(bin: string, args: string[]): Promise<{ code: number; stdout: string; stderr: string }> {
const cmd = new Deno.Command(bin, {
args,
stdin: "null",
stdout: "piped",
stderr: "piped",
});
try {
const { code, stdout, stderr } = await cmd.output();
const dec = new TextDecoder();
const result = {
code,
stdout: dec.decode(stdout),
stderr: dec.decode(stderr),
};
if (DOCKER_TEE) {
if (result.stdout.trim().length > 0) {
console.log(`[docker:${bin}] ${result.stdout.trimEnd()}`);
}
if (result.stderr.trim().length > 0) {
console.error(`[docker:${bin}] ${result.stderr.trimEnd()}`);
}
}
return result;
} catch (err) {
if (err instanceof Deno.errors.NotFound) {
return {
code: 127,
stdout: "",
stderr: `Command not found: ${bin}`,
};
}
throw err;
}
}
async function resolveDockerInvoker(): Promise<DockerInvoker> {
const custom = Deno.env.get("LIVESYNC_DOCKER_COMMAND")?.trim();
if (custom) {
const parsed = parseCommand(custom);
const runner: DockerInvoker = {
...parsed,
label: `custom(${custom})`,
};
// Validate custom command eagerly so misconfiguration fails fast.
const checkArgs = runner.prefix.length === 0 ? ["--version"] : [...runner.prefix, "docker", "--version"];
const check = await runCommand(runner.bin, checkArgs);
if (check.code !== 0) {
throw new Error(`LIVESYNC_DOCKER_COMMAND is not usable: ${custom}\n${check.stderr || check.stdout}`);
}
return runner;
}
const mode = (Deno.env.get("LIVESYNC_DOCKER_MODE") ?? "auto").toLowerCase();
const onWindows = Deno.build.os === "windows";
const native: DockerInvoker = { bin: "docker", prefix: [], label: "docker" };
const wsl: DockerInvoker = { bin: "wsl", prefix: [], label: "wsl docker" };
if (mode === "native") {
return native;
}
if (mode === "wsl") {
return wsl;
}
if (mode !== "auto") {
throw new Error(`Unsupported LIVESYNC_DOCKER_MODE='${mode}'. Use auto, native, or wsl.`);
}
// On Windows we prefer `wsl docker` first, then native docker.
// This typically works better in setups where Docker is installed only in
// WSL and not exposed as docker.exe on PATH.
const candidates = onWindows ? [wsl, native] : [native, wsl];
for (const c of candidates) {
if (c.bin === "docker") {
const r = await runCommand("docker", ["--version"]);
if (r.code === 0) return c;
continue;
}
const r = await runCommand("wsl", ["docker", "--version"]);
if (r.code === 0) return c;
}
throw new Error(
[
"Docker command is not available.",
"Set one of:",
"- LIVESYNC_DOCKER_MODE=native",
"- LIVESYNC_DOCKER_MODE=wsl",
"- LIVESYNC_DOCKER_COMMAND='docker'",
"- LIVESYNC_DOCKER_COMMAND='wsl docker'",
].join("\n")
);
}
async function getDockerInvoker(): Promise<DockerInvoker> {
if (!dockerInvokerPromise) {
dockerInvokerPromise = resolveDockerInvoker().then((r) => {
console.log(`[INFO] docker runner: ${r.label}`);
return r;
});
}
return await dockerInvokerPromise;
}
async function docker(...args: string[]): Promise<{ code: number; stdout: string; stderr: string }> {
const invoker = await getDockerInvoker();
// Either:
// docker <args>
// Or:
// wsl docker <args>
const finalArgs =
invoker.prefix.length === 0
? invoker.bin === "wsl"
? ["docker", ...args]
: args
: [...invoker.prefix, ...args];
const r = await runCommand(invoker.bin, finalArgs);
return { code: r.code, stdout: r.stdout, stderr: r.stderr };
}
async function dockerOrFail(...args: string[]): Promise<string> {
const r = await docker(...args);
if (r.code !== 0) {
throw new Error(`docker ${args[0]} failed (code ${r.code}): ${r.stderr.trim()}`);
}
return r.stdout;
}
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function waitForCouchdbStable(hostname: string, user: string, password: string): Promise<void> {
const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1");
const auth = btoa(`${user}:${password}`);
const headers = { Authorization: `Basic ${auth}` };
let consecutive = 0;
for (let i = 0; i < 30; i++) {
try {
const r = await fetch(`${h}/_up`, {
headers,
signal: AbortSignal.timeout(3000),
});
if (r.ok) {
consecutive++;
if (consecutive >= 3) return;
} else {
consecutive = 0;
}
} catch {
consecutive = 0;
}
await sleep(500);
}
throw new Error("CouchDB did not become stable in time");
}
// ---------------------------------------------------------------------------
// Fetch with retry (mirrors cli_test_curl_json() retry loop)
// ---------------------------------------------------------------------------
async function fetchRetry(
url: string,
init: RequestInit,
retries = 30,
delayMs = 2000,
allowStatus: number[] = []
): Promise<void> {
let lastError: unknown;
let lastStatus: number | undefined;
for (let i = 0; i < retries; i++) {
try {
const r = await fetch(url, {
signal: AbortSignal.timeout(5000),
...init,
});
lastStatus = r.status;
await r.body?.cancel().catch(() => {});
if (r.ok || allowStatus.includes(r.status)) return;
lastError = `HTTP ${r.status}`;
} catch (e) {
lastError = e;
}
await sleep(delayMs);
}
throw new Error(
`Could not reach ${url} after ${retries} retries: ${lastError} (last status: ${lastStatus ?? "N/A"})`
);
}
// ---------------------------------------------------------------------------
// CouchDB
// ---------------------------------------------------------------------------
//
// TODO: these values could be configurable via environment variables.
//
const COUCHDB_CONTAINER = "couchdb-test";
const COUCHDB_IMAGE = "couchdb:3.5.0";
const MINIO_CONTAINER = "minio-test";
const MINIO_IMAGE = "minio/minio";
const MINIO_MC_IMAGE = "minio/mc";
export async function stopCouchdb(): Promise<void> {
await docker("stop", COUCHDB_CONTAINER);
await docker("rm", COUCHDB_CONTAINER);
}
/**
* Start a CouchDB test container, initialise it, and create the test DB.
* Mirrors cli_test_start_couchdb() from test-helpers.sh, using direct
* docker / fetch calls instead of the bash util scripts.
*/
export async function startCouchdb(couchdbUri: string, user: string, password: string, dbname: string): Promise<void> {
console.log("[INFO] stopping leftover CouchDB container if present");
await stopCouchdb().catch(() => {});
console.log("[INFO] starting CouchDB test container");
await dockerOrFail(
"run",
"-d",
"--name",
COUCHDB_CONTAINER,
"-p",
// TODO: port mapping should be configurable.
"5989:5984",
"-e",
`COUCHDB_USER=${user}`,
"-e",
`COUCHDB_PASSWORD=${password}`,
"-e",
"COUCHDB_SINGLE_NODE=y",
COUCHDB_IMAGE
);
console.log("[INFO] initialising CouchDB");
await initCouchdb(couchdbUri, user, password);
console.log("[INFO] waiting for CouchDB to become stable");
await waitForCouchdbStable(couchdbUri, user, password);
console.log(`[INFO] creating test database: ${dbname}`);
await createCouchdbDatabase(couchdbUri, user, password, dbname);
}
/**
* Mirror couchdb-init.sh: configure single-node CouchDB via its REST API.
*/
async function initCouchdb(hostname: string, user: string, password: string, node = "_local"): Promise<void> {
// Podman environments often resolve localhost to ::1; use 127.0.0.1 like
// the bash script does.
const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1");
const auth = btoa(`${user}:${password}`);
const headers = {
"Content-Type": "application/json",
Authorization: `Basic ${auth}`,
};
const calls: Array<[string, string, string]> = [
[
"POST",
`${h}/_cluster_setup`,
JSON.stringify({
action: "enable_single_node",
username: user,
password,
bind_address: "0.0.0.0",
port: 5984,
singlenode: true,
}),
],
["PUT", `${h}/_node/${node}/_config/chttpd/require_valid_user`, '"true"'],
["PUT", `${h}/_node/${node}/_config/chttpd_auth/require_valid_user`, '"true"'],
["PUT", `${h}/_node/${node}/_config/httpd/WWW-Authenticate`, '"Basic realm=\\"couchdb\\""'],
["PUT", `${h}/_node/${node}/_config/httpd/enable_cors`, '"true"'],
["PUT", `${h}/_node/${node}/_config/chttpd/enable_cors`, '"true"'],
["PUT", `${h}/_node/${node}/_config/chttpd/max_http_request_size`, '"4294967296"'],
["PUT", `${h}/_node/${node}/_config/couchdb/max_document_size`, '"50000000"'],
["PUT", `${h}/_node/${node}/_config/cors/credentials`, '"true"'],
["PUT", `${h}/_node/${node}/_config/cors/origins`, '"*"'],
];
for (const [method, url, body] of calls) {
await fetchRetry(url, { method, headers, body });
}
}
export async function createCouchdbDatabase(
hostname: string,
user: string,
password: string,
dbname: string
): Promise<void> {
const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1");
const auth = btoa(`${user}:${password}`);
await fetchRetry(`${h}/${dbname}`, {
method: "PUT",
headers: { Authorization: `Basic ${auth}` },
});
}
/** Update a CouchDB document via PUT. Returns the updated document. */
export async function updateCouchdbDoc(
hostname: string,
user: string,
password: string,
docUrl: string,
updater: (doc: Record<string, unknown>) => Record<string, unknown>
): Promise<void> {
const h = hostname.replace(/\/$/, "").replace("localhost", "127.0.0.1");
const auth = btoa(`${user}:${password}`);
const headers = {
"Content-Type": "application/json",
Authorization: `Basic ${auth}`,
};
const getRes = await fetch(`${h}/${docUrl}`, { headers });
const current = (await getRes.json()) as Record<string, unknown>;
const updated = updater(current);
await fetchRetry(`${h}/${docUrl}`, {
method: "PUT",
headers,
body: JSON.stringify(updated),
});
}
// ---------------------------------------------------------------------------
// MinIO
// ---------------------------------------------------------------------------
function shQuote(value: string): string {
return `'${value.split("'").join(`'"'"'`)}'`;
}
export async function stopMinio(): Promise<void> {
await docker("stop", MINIO_CONTAINER);
await docker("rm", MINIO_CONTAINER);
}
async function initMinioBucket(
minioEndpoint: string,
accessKey: string,
secretKey: string,
bucket: string
): Promise<boolean> {
const cmd =
`mc alias set myminio ${shQuote(minioEndpoint)} ${shQuote(accessKey)} ${shQuote(secretKey)} >/dev/null 2>&1 && ` +
`mc mb --ignore-existing myminio/${shQuote(bucket)} >/dev/null 2>&1`;
const r = await docker("run", "--rm", "--network", "host", "--entrypoint", "/bin/sh", MINIO_MC_IMAGE, "-c", cmd);
return r.code === 0;
}
async function waitForMinioBucket(
minioEndpoint: string,
accessKey: string,
secretKey: string,
bucket: string
): Promise<void> {
for (let i = 0; i < 30; i++) {
const checkCmd =
`mc alias set myminio ${shQuote(minioEndpoint)} ${shQuote(accessKey)} ${shQuote(secretKey)} >/dev/null 2>&1 && ` +
`mc ls myminio/${shQuote(bucket)} >/dev/null 2>&1`;
const check = await docker(
"run",
"--rm",
"--network",
// Now I used host networking to access the container via localhost for some environments (Docker Desktop on Windows).
// We need something good idea to work across all environments.
"host",
"--entrypoint",
"/bin/sh",
MINIO_MC_IMAGE,
"-c",
checkCmd
);
if (check.code === 0) {
return;
}
await initMinioBucket(minioEndpoint, accessKey, secretKey, bucket);
await sleep(2000);
}
throw new Error(`MinIO bucket not ready: ${bucket}`);
}
export async function startMinio(
minioEndpoint: string,
accessKey: string,
secretKey: string,
bucket: string
): Promise<void> {
console.log("[INFO] stopping leftover MinIO container if present");
await stopMinio().catch(() => {});
console.log("[INFO] starting MinIO test container");
await dockerOrFail(
"run",
"-d",
"--name",
MINIO_CONTAINER,
// TODO: Ports should be configurable.
"-p",
"9000:9000",
"-p",
"9001:9001",
"-e",
`MINIO_ROOT_USER=${accessKey}`,
"-e",
`MINIO_ROOT_PASSWORD=${secretKey}`,
"-e",
`MINIO_SERVER_URL=${minioEndpoint}`,
MINIO_IMAGE,
"server",
"/data",
"--console-address",
":9001"
);
console.log(`[INFO] initialising MinIO test bucket: ${bucket}`);
let initialised = false;
for (let i = 0; i < 5; i++) {
if (await initMinioBucket(minioEndpoint, accessKey, secretKey, bucket)) {
initialised = true;
break;
}
await sleep(2000);
}
if (!initialised) {
throw new Error(`Could not initialise MinIO bucket after retries: ${bucket}`);
}
await waitForMinioBucket(minioEndpoint, accessKey, secretKey, bucket);
}
// ---------------------------------------------------------------------------
// P2P relay (strfry)
// ---------------------------------------------------------------------------
// TODO: these values could be configurable via environment variables.
const P2P_RELAY_CONTAINER = "relay-test";
const P2P_RELAY_IMAGE = "ghcr.io/hoytech/strfry:latest";
const STRFRY_BOOTSTRAP_SH = String.raw`cat > /tmp/strfry.conf <<"EOF"
db = "./strfry-db/"
relay {
bind = "0.0.0.0"
port = 7777
nofiles = 100000
info {
name = "livesync test relay"
description = "local relay for livesync p2p tests"
}
maxWebsocketPayloadSize = 131072
autoPingSeconds = 55
writePolicy {
plugin = ""
}
}
EOF
exec /app/strfry --config /tmp/strfry.conf relay`;
export async function stopP2pRelay(): Promise<void> {
await docker("stop", P2P_RELAY_CONTAINER);
await docker("rm", P2P_RELAY_CONTAINER);
}
/**
* Start the local P2P relay container through the same docker runner used
* by CouchDB helpers. This keeps process ownership consistent across
* start/stop on Windows, WSL, and native Linux/macOS.
*/
export async function startP2pRelay(): Promise<void> {
console.log("[INFO] stopping leftover P2P relay container if present");
await stopP2pRelay().catch(() => {});
console.log("[INFO] starting local P2P relay container");
await dockerOrFail(
"run",
"-d",
"--name",
P2P_RELAY_CONTAINER,
"-p",
//TODO: port mapping should be configurable.
"4000:7777",
"--tmpfs",
"/app/strfry-db:rw,size=256m",
"--entrypoint",
"sh",
P2P_RELAY_IMAGE,
"-lc",
STRFRY_BOOTSTRAP_SH
);
}
export function isLocalP2pRelay(relayUrl: string): boolean {
return relayUrl === "ws://localhost:4000" || relayUrl === "ws://localhost:4000/";
}

View File

@@ -0,0 +1,26 @@
/**
* Load a .env-style file (KEY=value per line) into a plain object.
* Equivalent to `source $TEST_ENV_FILE; set -a` in bash.
* Maybe we should use some library... now it is just the minimal implementation that covers our use cases.
*
* Supported value formats:
* KEY=value
* KEY='single quoted'
* KEY="double quoted"
* # comment lines are ignored
*/
export async function loadEnvFile(filePath: string): Promise<Record<string, string>> {
const text = await Deno.readTextFile(filePath);
const result: Record<string, string> = {};
for (const line of text.split("\n")) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith("#")) continue;
const idx = trimmed.indexOf("=");
if (idx < 0) continue;
const key = trimmed.slice(0, idx).trim();
const raw = trimmed.slice(idx + 1).trim();
// Strip surrounding single or double quotes
result[key] = raw.replace(/^(['"])(.*)\1$/, "$2");
}
return result;
}

View File

@@ -0,0 +1,52 @@
import { runCli } from "./cli.ts";
import { isLocalP2pRelay, startP2pRelay, stopP2pRelay } from "./docker.ts";
export type PeerEntry = {
id: string;
name: string;
};
export function parsePeerLines(output: string): PeerEntry[] {
return output
.split(/\r?\n/)
.map((line) => line.split("\t"))
.filter((parts) => parts.length >= 3 && parts[0] === "[peer]")
.map((parts) => ({ id: parts[1], name: parts[2] }));
}
export async function discoverPeer(
vaultDir: string,
settingsFile: string,
timeoutSeconds: number,
targetPeer?: string
): Promise<PeerEntry> {
const result = await runCli(vaultDir, "--settings", settingsFile, "p2p-peers", String(timeoutSeconds));
if (result.code !== 0) {
throw new Error(`p2p-peers failed\n${result.combined}`);
}
const peers = parsePeerLines(result.stdout);
if (targetPeer) {
const matched = peers.find((peer) => peer.id === targetPeer || peer.name === targetPeer);
if (matched) return matched;
}
if (peers.length === 0) {
const fallback = result.combined.match(/Advertisement from\s+([^\s]+)/);
if (fallback?.[1]) {
return { id: fallback[1], name: fallback[1] };
}
throw new Error(`No peers discovered\n${result.combined}`);
}
return peers[0];
}
export async function maybeStartLocalRelay(relay: string): Promise<boolean> {
if (!isLocalP2pRelay(relay)) return false;
await startP2pRelay();
return true;
}
export async function stopLocalRelayIfStarted(started: boolean): Promise<void> {
if (started) {
await stopP2pRelay().catch(() => {});
}
}

View File

@@ -0,0 +1,205 @@
import { join } from "@std/path";
import { CLI_DIR, runCliOrFail } from "./cli.ts";
// ---------------------------------------------------------------------------
// Settings file initialisation
// ---------------------------------------------------------------------------
/** Generate a default settings file using the CLI's init-settings command. */
export async function initSettingsFile(settingsFile: string): Promise<void> {
await runCliOrFail("init-settings", "--force", settingsFile);
}
/**
* Generate a full setup URI from a settings file via src/lib API.
* Mirrors the bash flow in test-setup-put-cat-linux.sh.
*/
export async function generateSetupUriFromSettings(settingsFile: string, setupPassphrase: string): Promise<string> {
const repoRoot = join(CLI_DIR, "..", "..", "..");
const script = [
"import fs from 'node:fs';",
"import { pathToFileURL } from 'node:url';",
"(async () => {",
" const modulePath = process.env.REPO_ROOT + '/src/lib/src/API/processSetting.ts';",
" const moduleUrl = pathToFileURL(modulePath).href;",
" const { encodeSettingsToSetupURI } = await import(moduleUrl);",
" const settingsPath = process.env.SETTINGS_FILE;",
" const passphrase = process.env.SETUP_PASSPHRASE;",
" const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf-8'));",
" settings.couchDB_DBNAME = 'setup-put-cat-db';",
" settings.couchDB_URI = 'http://127.0.0.1:5999';",
" settings.couchDB_USER = 'dummy';",
" settings.couchDB_PASSWORD = 'dummy';",
" settings.liveSync = false;",
" settings.syncOnStart = false;",
" settings.syncOnSave = false;",
" const uri = await encodeSettingsToSetupURI(settings, passphrase);",
" process.stdout.write(uri.trim());",
"})();",
].join("\n");
const scriptPath = await Deno.makeTempFile({
prefix: "livesync-setup-uri-",
suffix: ".mts",
});
await Deno.writeTextFile(scriptPath, script);
try {
const cmd = new Deno.Command("npx", {
args: ["tsx", scriptPath],
cwd: CLI_DIR,
env: {
REPO_ROOT: repoRoot,
SETTINGS_FILE: settingsFile,
SETUP_PASSPHRASE: setupPassphrase,
},
stdin: "null",
stdout: "piped",
stderr: "piped",
});
const { code, stdout, stderr } = await cmd.output();
const dec = new TextDecoder();
if (code !== 0) {
throw new Error(
`Failed to generate setup URI (code ${code})\nstdout: ${dec.decode(stdout)}\nstderr: ${dec.decode(stderr)}`
);
}
const uri = dec.decode(stdout).trim();
if (!uri) {
throw new Error("Failed to generate setup URI: output is empty");
}
return uri;
} finally {
await Deno.remove(scriptPath).catch(() => {});
}
}
/** Set isConfigured=true in a settings file (required for mirror / scan). */
export async function markSettingsConfigured(settingsFile: string): Promise<void> {
const data = JSON.parse(await Deno.readTextFile(settingsFile));
data.isConfigured = true;
await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2));
}
// ---------------------------------------------------------------------------
// CouchDB remote settings
// ---------------------------------------------------------------------------
/**
* Apply CouchDB connection details to a settings file.
* Mirrors cli_test_apply_couchdb_settings() from test-helpers.sh.
*/
export async function applyCouchdbSettings(
settingsFile: string,
couchdbUri: string,
couchdbUser: string,
couchdbPassword: string,
couchdbDbname: string,
liveSync = false
): Promise<void> {
const data = JSON.parse(await Deno.readTextFile(settingsFile));
data.couchDB_URI = couchdbUri;
data.couchDB_USER = couchdbUser;
data.couchDB_PASSWORD = couchdbPassword;
data.couchDB_DBNAME = couchdbDbname;
if (liveSync) {
data.liveSync = true;
data.syncOnStart = false;
data.syncOnSave = false;
data.usePluginSync = false;
}
data.isConfigured = true;
await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2));
}
export async function applyRemoteSyncSettings(
settingsFile: string,
options: {
remoteType: "COUCHDB" | "MINIO";
couchdbUri?: string;
couchdbUser?: string;
couchdbPassword?: string;
couchdbDbname?: string;
minioBucket?: string;
minioEndpoint?: string;
minioAccessKey?: string;
minioSecretKey?: string;
encrypt?: boolean;
passphrase?: string;
}
): Promise<void> {
const data = JSON.parse(await Deno.readTextFile(settingsFile));
if (options.remoteType === "COUCHDB") {
data.remoteType = "";
data.couchDB_URI = options.couchdbUri;
data.couchDB_USER = options.couchdbUser;
data.couchDB_PASSWORD = options.couchdbPassword;
data.couchDB_DBNAME = options.couchdbDbname;
} else {
data.remoteType = "MINIO";
data.bucket = options.minioBucket;
data.endpoint = options.minioEndpoint;
data.accessKey = options.minioAccessKey;
data.secretKey = options.minioSecretKey;
data.region = "auto";
data.forcePathStyle = true;
}
data.liveSync = true;
data.syncOnStart = false;
data.syncOnSave = false;
data.usePluginSync = false;
data.encrypt = options.encrypt === true;
data.passphrase = options.encrypt ? (options.passphrase ?? "") : "";
data.isConfigured = true;
await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2));
}
// ---------------------------------------------------------------------------
// P2P settings
// ---------------------------------------------------------------------------
/**
* Apply P2P connection details to a settings file.
* Mirrors cli_test_apply_p2p_settings() from test-helpers.sh.
*/
export async function applyP2pSettings(
settingsFile: string,
roomId: string,
passphrase: string,
appId = "self-hosted-livesync-cli-tests",
relays = "ws://localhost:4000/",
autoAccept = "~.*"
): Promise<void> {
const data = JSON.parse(await Deno.readTextFile(settingsFile));
data.P2P_Enabled = true;
data.P2P_AutoStart = false;
data.P2P_AutoBroadcast = false;
data.P2P_AppID = appId;
data.P2P_roomID = roomId;
data.P2P_passphrase = passphrase;
data.P2P_relays = relays;
data.P2P_AutoAcceptingPeers = autoAccept;
data.P2P_AutoDenyingPeers = "";
data.P2P_IsHeadless = true;
data.isConfigured = true;
await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2));
}
export async function applyP2pTestTweaks(settingsFile: string, deviceName: string, passphrase: string): Promise<void> {
const data = JSON.parse(await Deno.readTextFile(settingsFile));
data.remoteType = "ONLY_P2P";
data.encrypt = true;
data.passphrase = passphrase;
data.usePathObfuscation = true;
data.handleFilenameCaseSensitive = false;
data.customChunkSize = 50;
data.usePluginSyncV2 = true;
data.doNotUseFixedRevisionForChunks = false;
data.P2P_DevicePeerName = deviceName;
data.isConfigured = true;
await Deno.writeTextFile(settingsFile, JSON.stringify(data, null, 2));
}

View File

@@ -0,0 +1,33 @@
import { join } from "@std/path";
/**
* A temporary directory that cleans itself up via `await using`.
* Requires TypeScript 5.2+ / Deno 1.40+ for the AsyncDisposable protocol.
*
* @example
* ```ts
* await using tmp = await TempDir.create();
* const file = tmp.join("data.json");
* ```
*/
export class TempDir implements AsyncDisposable {
readonly path: string;
private constructor(path: string) {
this.path = path;
}
static async create(prefix = "livesync-deno-test"): Promise<TempDir> {
const path = await Deno.makeTempDir({ prefix: `${prefix}.` });
return new TempDir(path);
}
/** Return an OS path joined to the temp directory root. */
join(...parts: string[]): string {
return join(this.path, ...parts);
}
async [Symbol.asyncDispose](): Promise<void> {
await Deno.remove(this.path, { recursive: true }).catch(() => {});
}
}

View File

@@ -0,0 +1,277 @@
import { assert } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import {
runCli,
runCliOrFail,
runCliWithInputOrFail,
sanitiseCatStdout,
assertFilesEqual,
jsonStringField,
} from "./helpers/cli.ts";
import { applyRemoteSyncSettings, initSettingsFile } from "./helpers/settings.ts";
import { startCouchdb, startMinio, stopCouchdb, stopMinio } from "./helpers/docker.ts";
type RemoteType = "COUCHDB" | "MINIO";
function requireEnv(...keys: string[]): string {
for (const key of keys) {
const value = Deno.env.get(key)?.trim();
if (value) return value;
}
throw new Error(`Required env var is missing: ${keys.join(" or ")}`);
}
export async function runScenario(remoteType: RemoteType, encrypt: boolean): Promise<void> {
const dbSuffix = `${Date.now()}-${Math.floor(Math.random() * 100000)}`;
const couchdbUri = remoteType === "COUCHDB" ? requireEnv("COUCHDB_URI", "hostname").replace(/\/$/, "") : "";
const couchdbUser = remoteType === "COUCHDB" ? requireEnv("COUCHDB_USER", "username") : "";
const couchdbPassword = remoteType === "COUCHDB" ? requireEnv("COUCHDB_PASSWORD", "password") : "";
const dbPrefix = remoteType === "COUCHDB" ? requireEnv("COUCHDB_DBNAME", "dbname") : "";
const dbname = remoteType === "COUCHDB" ? `${dbPrefix}-${dbSuffix}` : "";
const minioEndpoint =
remoteType === "MINIO" ? requireEnv("MINIO_ENDPOINT", "minioEndpoint").replace(/\/$/, "") : "";
const minioAccessKey = remoteType === "MINIO" ? requireEnv("MINIO_ACCESS_KEY", "accessKey") : "";
const minioSecretKey = remoteType === "MINIO" ? requireEnv("MINIO_SECRET_KEY", "secretKey") : "";
const minioBucketBase = remoteType === "MINIO" ? requireEnv("MINIO_BUCKET_NAME", "bucketName") : "";
const minioBucket = remoteType === "MINIO" ? `${minioBucketBase}-${dbSuffix}` : "";
const passphrase = "e2e-passphrase";
await using workDir = await TempDir.create(
`livesync-cli-e2e-${remoteType.toLowerCase()}-${encrypt ? "enc1" : "enc0"}`
);
const vaultA = workDir.join("testvault_a");
const vaultB = workDir.join("testvault_b");
const settingsA = workDir.join("test-settings-a.json");
const settingsB = workDir.join("test-settings-b.json");
const pushSrc = workDir.join("push-source.txt");
const pullDst = workDir.join("pull-destination.txt");
const pushBinarySrc = workDir.join("push-source.bin");
const pullBinaryDst = workDir.join("pull-destination.bin");
await Deno.mkdir(vaultA, { recursive: true });
await Deno.mkdir(vaultB, { recursive: true });
const keepDocker = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1";
if (remoteType === "COUCHDB") {
await startCouchdb(couchdbUri, couchdbUser, couchdbPassword, dbname);
} else {
await startMinio(minioEndpoint, minioAccessKey, minioSecretKey, minioBucket);
}
try {
await initSettingsFile(settingsA);
await initSettingsFile(settingsB);
await applyRemoteSyncSettings(settingsA, {
remoteType,
couchdbUri,
couchdbUser,
couchdbPassword,
couchdbDbname: dbname,
minioBucket,
minioEndpoint,
minioAccessKey,
minioSecretKey,
encrypt,
passphrase,
});
await applyRemoteSyncSettings(settingsB, {
remoteType,
couchdbUri,
couchdbUser,
couchdbPassword,
couchdbDbname: dbname,
minioBucket,
minioEndpoint,
minioAccessKey,
minioSecretKey,
encrypt,
passphrase,
});
const syncBoth = async () => {
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
};
const targetAOnly = "e2e/a-only-info.md";
const targetSync = "e2e/sync-info.md";
const targetSyncTwiceFirst = "e2e/sync-twice-first.md";
const targetSyncTwiceSecond = "e2e/sync-twice-second.md";
const targetPush = "e2e/pushed-from-a.md";
const targetPut = "e2e/put-from-a.md";
const targetPushBinary = "e2e/pushed-from-a.bin";
const targetConflict = "e2e/conflict.md";
await runCliWithInputOrFail("alpha-from-a\n", vaultA, "--settings", settingsA, "put", targetAOnly);
const infoAOnly = await runCliOrFail(vaultA, "--settings", settingsA, "info", targetAOnly);
assert(infoAOnly.includes(`"path": "${targetAOnly}"`));
await runCliWithInputOrFail("visible-after-sync\n", vaultA, "--settings", settingsA, "put", targetSync);
await syncBoth();
const infoBSync = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetSync);
assert(infoBSync.includes(`"path": "${targetSync}"`));
await runCliWithInputOrFail(
`first-sync-round-${dbSuffix}\n`,
vaultA,
"--settings",
settingsA,
"put",
targetSyncTwiceFirst
);
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
const firstVisible = sanitiseCatStdout(
await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetSyncTwiceFirst)
).trimEnd();
assert(firstVisible === `first-sync-round-${dbSuffix}`);
await runCliWithInputOrFail(
`second-sync-round-${dbSuffix}\n`,
vaultA,
"--settings",
settingsA,
"put",
targetSyncTwiceSecond
);
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
const secondVisible = sanitiseCatStdout(
await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetSyncTwiceSecond)
).trimEnd();
assert(secondVisible === `second-sync-round-${dbSuffix}`);
await Deno.writeTextFile(pushSrc, `pushed-content-${dbSuffix}\n`);
await runCliOrFail(vaultA, "--settings", settingsA, "push", pushSrc, targetPush);
await runCliWithInputOrFail(`put-content-${dbSuffix}\n`, vaultA, "--settings", settingsA, "put", targetPut);
await syncBoth();
await runCliOrFail(vaultB, "--settings", settingsB, "pull", targetPush, pullDst);
await assertFilesEqual(pushSrc, pullDst, "B pull result does not match pushed source");
const catBPut = sanitiseCatStdout(
await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetPut)
).trimEnd();
assert(catBPut === `put-content-${dbSuffix}`);
const binary = new Uint8Array(4096);
binary.fill(0x61);
await Deno.writeFile(pushBinarySrc, binary);
await runCliOrFail(vaultA, "--settings", settingsA, "push", pushBinarySrc, targetPushBinary);
await syncBoth();
await runCliOrFail(vaultB, "--settings", settingsB, "pull", targetPushBinary, pullBinaryDst);
await assertFilesEqual(pushBinarySrc, pullBinaryDst, "B pull result does not match pushed binary source");
await runCliOrFail(vaultA, "--settings", settingsA, "rm", targetPut);
await syncBoth();
const removed = await runCli(vaultB, "--settings", settingsB, "cat", targetPut);
assert(removed.code !== 0, `B cat should fail after A removed the file\n${removed.combined}`);
await runCliWithInputOrFail("conflict-base\n", vaultA, "--settings", settingsA, "put", targetConflict);
await syncBoth();
await runCliWithInputOrFail(
`conflict-from-a-${dbSuffix}\n`,
vaultA,
"--settings",
settingsA,
"put",
targetConflict
);
await runCliWithInputOrFail(
`conflict-from-b-${dbSuffix}\n`,
vaultB,
"--settings",
settingsB,
"put",
targetConflict
);
let infoAConflict = "";
let infoBConflict = "";
let conflictDetected = false;
for (const side of ["a", "b", "a"] as const) {
await runCliOrFail(
side === "a" ? vaultA : vaultB,
"--settings",
side === "a" ? settingsA : settingsB,
"sync"
);
infoAConflict = await runCliOrFail(vaultA, "--settings", settingsA, "info", targetConflict);
infoBConflict = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetConflict);
if (
jsonStringField(infoAConflict, "conflicts") !== "N/A" ||
jsonStringField(infoBConflict, "conflicts") !== "N/A"
) {
conflictDetected = true;
break;
}
}
assert(conflictDetected, `conflict was expected\nA: ${infoAConflict}\nB: ${infoBConflict}`);
const lsAConflict =
(await runCliOrFail(vaultA, "--settings", settingsA, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? "";
const lsBConflict =
(await runCliOrFail(vaultB, "--settings", settingsB, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? "";
const revA = lsAConflict.split("\t")[3] ?? "";
const revB = lsBConflict.split("\t")[3] ?? "";
assert(
revA.includes("*") || revB.includes("*"),
`conflicted entry should be marked with '*'\nA: ${lsAConflict}\nB: ${lsBConflict}`
);
const keepRevision = jsonStringField(infoAConflict, "revision");
assert(keepRevision.length > 0, `could not extract revision\n${infoAConflict}`);
await runCliOrFail(vaultA, "--settings", settingsA, "resolve", targetConflict, keepRevision);
let resolved = false;
let infoAResolved = "";
let infoBResolved = "";
for (let i = 0; i < 6; i++) {
await syncBoth();
infoAResolved = await runCliOrFail(vaultA, "--settings", settingsA, "info", targetConflict);
infoBResolved = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetConflict);
if (
jsonStringField(infoAResolved, "conflicts") === "N/A" &&
jsonStringField(infoBResolved, "conflicts") === "N/A"
) {
resolved = true;
break;
}
const retryRevision = jsonStringField(infoAResolved, "revision");
if (retryRevision) {
await runCli(vaultA, "--settings", settingsA, "resolve", targetConflict, retryRevision);
}
}
assert(resolved, `conflicts should be resolved\nA: ${infoAResolved}\nB: ${infoBResolved}`);
const lsAResolved =
(await runCliOrFail(vaultA, "--settings", settingsA, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? "";
const lsBResolved =
(await runCliOrFail(vaultB, "--settings", settingsB, "ls", targetConflict)).trim().split(/\r?\n/)[0] ?? "";
assert(!(lsAResolved.split("\t")[3] ?? "").includes("*"));
assert(!(lsBResolved.split("\t")[3] ?? "").includes("*"));
const catAResolved = sanitiseCatStdout(
await runCliOrFail(vaultA, "--settings", settingsA, "cat", targetConflict)
).trimEnd();
const catBResolved = sanitiseCatStdout(
await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetConflict)
).trimEnd();
assert(catAResolved === catBResolved, `resolved content should match\nA: ${catAResolved}\nB: ${catBResolved}`);
} finally {
if (!keepDocker) {
if (remoteType === "COUCHDB") {
await stopCouchdb().catch(() => {});
} else {
await stopMinio().catch(() => {});
}
}
}
}
Deno.test("e2e: two vaults over CouchDB without encryption", async () => {
await runScenario("COUCHDB", false);
});
Deno.test("e2e: two vaults over CouchDB with encryption", async () => {
await runScenario("COUCHDB", true);
});

View File

@@ -0,0 +1,20 @@
import { runScenario } from "./test-e2e-two-vaults-couchdb.ts";
type MatrixCase = {
remoteType: "COUCHDB" | "MINIO";
encrypt: boolean;
label: string;
};
const matrixCases: MatrixCase[] = [
{ remoteType: "COUCHDB", encrypt: false, label: "COUCHDB-enc0" },
{ remoteType: "COUCHDB", encrypt: true, label: "COUCHDB-enc1" },
{ remoteType: "MINIO", encrypt: false, label: "MINIO-enc0" },
{ remoteType: "MINIO", encrypt: true, label: "MINIO-enc1" },
];
for (const tc of matrixCases) {
Deno.test(`e2e matrix: ${tc.label}`, async () => {
await runScenario(tc.remoteType, tc.encrypt);
});
}

View File

@@ -0,0 +1,196 @@
/**
* Deno port of test-mirror-linux.sh
*
* Tests the `mirror` command — bidirectional synchronisation between a local
* storage directory (vault) and an in-process database.
*
* Covered cases (identical to the bash test):
* 1. Storage-only file -> synced into DB (UPDATE DATABASE)
* 2. DB-only file -> restored to storage (UPDATE STORAGE)
* 3. DB-deleted file -> NOT restored to storage (UPDATE STORAGE skip)
* 4. Both, storage newer -> DB updated (SYNC: STORAGE -> DB)
* 5. Both, DB newer -> storage updated (SYNC: DB -> STORAGE)
* 6. Compatibility mode -> omitted vault-path works (same DB + vault path)
*
* No external services are required.
*
* Run:
* deno test -A test-mirror.ts
*/
import { assert } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { runCliOrFail } from "./helpers/cli.ts";
import { initSettingsFile, markSettingsConfigured } from "./helpers/settings.ts";
Deno.test("mirror: storage <-> DB synchronisation", async (t) => {
await using workDir = await TempDir.create("livesync-cli-mirror");
// -------------------------------------------------------------------
// Shared setup
// -------------------------------------------------------------------
const settingsFile = workDir.join("data.json");
const vaultDir = workDir.join("vault");
const dbDir = workDir.join("db");
await Deno.mkdir(workDir.join("vault", "test"), { recursive: true });
await Deno.mkdir(dbDir, { recursive: true });
await initSettingsFile(settingsFile);
// isConfigured=true is required for canProceedScan in the mirror command.
await markSettingsConfigured(settingsFile);
// Copy settings to the DB directory (separated-path mode)
const dbSettings = workDir.join("db", "settings.json");
await Deno.copyFile(settingsFile, dbSettings);
/** Run mirror in separated-path mode: DB dir ≠ vault dir. */
const runMirror = () => runCliOrFail(dbDir, "--settings", dbSettings, "mirror", vaultDir);
/** Run mirror in compatibility mode: DB path = vault path. */
const runMirrorCompat = () => runCliOrFail(vaultDir, "--settings", settingsFile, "mirror");
// Helper wrappers
const dbRun = (...args: string[]) => runCliOrFail(dbDir, "--settings", dbSettings, ...args);
const compatRun = (...args: string[]) => runCliOrFail(vaultDir, "--settings", settingsFile, ...args);
// -------------------------------------------------------------------
// Case 1: storage-only -> DB (UPDATE DATABASE)
// -------------------------------------------------------------------
await t.step("case 1: storage-only file is synced into DB", async () => {
const storageFile = workDir.join("vault", "test", "storage-only.md");
await Deno.writeTextFile(storageFile, "storage-only content\n");
await runMirror();
const resultFile = workDir.join("case1-pull.txt");
await dbRun("pull", "test/storage-only.md", resultFile);
const storageContent = await Deno.readTextFile(storageFile);
const pulledContent = await Deno.readTextFile(resultFile);
assert(
storageContent === pulledContent,
`storage-only file NOT synced into DB\nexpected: ${storageContent}\ngot: ${pulledContent}`
);
console.log("[PASS] case 1: storage-only file was synced into DB");
});
// -------------------------------------------------------------------
// Case 2: DB-only -> storage (UPDATE STORAGE)
// -------------------------------------------------------------------
await t.step("case 2: DB-only file is restored to storage", async () => {
await dbRun(
"push",
// write inline via push (pipe not needed — push takes a file path)
// create a temp file with content and push it
await (async () => {
const tmp = workDir.join("db-only-src.txt");
await Deno.writeTextFile(tmp, "db-only content\n");
return tmp;
})(),
"test/db-only.md"
);
const storagePath = workDir.join("vault", "test", "db-only.md");
assert(!(await exists(storagePath)), "db-only.md unexpectedly exists in storage before mirror");
await runMirror();
assert(await exists(storagePath), "DB-only file NOT restored to storage after mirror");
const content = await Deno.readTextFile(storagePath);
assert(content === "db-only content\n", `DB-only file restored but content mismatch: '${content}'`);
console.log("[PASS] case 2: DB-only file was restored to storage");
});
// -------------------------------------------------------------------
// Case 3: DB-deleted -> storage untouched
// -------------------------------------------------------------------
await t.step("case 3: DB-deleted entry is NOT restored to storage", async () => {
const deletedSrc = workDir.join("deleted-src.txt");
await Deno.writeTextFile(deletedSrc, "to-be-deleted\n");
await dbRun("push", deletedSrc, "test/deleted.md");
await dbRun("rm", "test/deleted.md");
await runMirror();
const storagePath = workDir.join("vault", "test", "deleted.md");
assert(!(await exists(storagePath)), "deleted DB entry was incorrectly restored to storage");
console.log("[PASS] case 3: deleted DB entry was NOT restored to storage");
});
// -------------------------------------------------------------------
// Case 4: storage newer -> DB updated (SYNC: STORAGE -> DB)
// -------------------------------------------------------------------
await t.step("case 4: storage newer than DB -> DB is updated", async () => {
// Seed DB with old content (mtime ~ now)
const seedFile = workDir.join("case4-seed.txt");
await Deno.writeTextFile(seedFile, "old content\n");
await dbRun("push", seedFile, "test/sync-storage-newer.md");
// Write new content to storage with a timestamp 1 hour in the future
const storageFile = workDir.join("vault", "test", "sync-storage-newer.md");
await Deno.writeTextFile(storageFile, "new content\n");
await Deno.utime(storageFile, new Date(), new Date(Date.now() + 3600_000));
await runMirror();
const resultFile = workDir.join("case4-pull.txt");
await dbRun("pull", "test/sync-storage-newer.md", resultFile);
const storageContent = await Deno.readTextFile(storageFile);
const pulledContent = await Deno.readTextFile(resultFile);
assert(
storageContent === pulledContent,
`DB NOT updated to match newer storage file\nexpected: ${storageContent}\ngot: ${pulledContent}`
);
console.log("[PASS] case 4: DB updated to match newer storage file");
});
// -------------------------------------------------------------------
// Case 5: DB newer -> storage updated (SYNC: DB -> STORAGE)
// -------------------------------------------------------------------
await t.step("case 5: DB newer than storage -> storage is updated", async () => {
// Write old content to storage with a timestamp 1 hour in the past
const storageFile = workDir.join("vault", "test", "sync-db-newer.md");
await Deno.writeTextFile(storageFile, "old storage content\n");
await Deno.utime(storageFile, new Date(), new Date(Date.now() - 3600_000));
// Write new content to DB only (mtime ~ now, newer than the storage file)
const dbNewFile = workDir.join("case5-db-new.txt");
await Deno.writeTextFile(dbNewFile, "new db content\n");
await dbRun("push", dbNewFile, "test/sync-db-newer.md");
await runMirror();
const content = await Deno.readTextFile(storageFile);
assert(content === "new db content\n", `storage NOT updated to match newer DB entry (got: '${content}')`);
console.log("[PASS] case 5: storage updated to match newer DB entry");
});
// -------------------------------------------------------------------
// Case 6: compatibility mode (vault path = DB path)
// -------------------------------------------------------------------
await t.step("case 6: compatibility mode (omitted vault-path)", async () => {
const compatFile = workDir.join("vault", "compat.md");
await Deno.writeTextFile(compatFile, "compat-content\n");
await runMirrorCompat();
const resultFile = workDir.join("case6-pull.txt");
await compatRun("pull", "compat.md", resultFile);
const pulled = await Deno.readTextFile(resultFile);
assert(pulled === "compat-content\n", `Compatibility mode failed to sync file into DB (got: '${pulled}')`);
console.log("[PASS] case 6: compatibility mode works");
});
});
// ---------------------------------------------------------------------------
// Utility
// ---------------------------------------------------------------------------
async function exists(path: string): Promise<boolean> {
try {
await Deno.stat(path);
return true;
} catch {
return false;
}
}

View File

@@ -0,0 +1,40 @@
import { assert } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { initSettingsFile, applyP2pSettings } from "./helpers/settings.ts";
import { startP2pRelay, stopP2pRelay, isLocalP2pRelay } from "./helpers/docker.ts";
import { startCliInBackground } from "./helpers/backgroundCli.ts";
Deno.test("p2p-host: starts and becomes ready", async () => {
const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/";
const roomId = Deno.env.get("ROOM_ID") ?? `room-${Date.now()}`;
const passphrase = Deno.env.get("PASSPHRASE") ?? "test";
const appId = Deno.env.get("APP_ID") ?? "self-hosted-livesync-cli-tests";
const useInternalRelay = Deno.env.get("USE_INTERNAL_RELAY") !== "0";
await using workDir = await TempDir.create("livesync-cli-p2p-host");
const vaultDir = workDir.join("vault-host");
const settingsFile = workDir.join("settings-host.json");
await Deno.mkdir(vaultDir, { recursive: true });
let relayStarted = false;
if (useInternalRelay && isLocalP2pRelay(relay)) {
await startP2pRelay();
relayStarted = true;
}
try {
await initSettingsFile(settingsFile);
await applyP2pSettings(settingsFile, roomId, passphrase, appId, relay);
const host = startCliInBackground(vaultDir, "--settings", settingsFile, "p2p-host");
try {
await host.waitUntilContains("P2P host is running", 20000);
assert(host.combined.includes("P2P host is running"));
} finally {
await host.stop();
}
} finally {
if (relayStarted) {
await stopP2pRelay().catch(() => {});
}
}
});

View File

@@ -0,0 +1,42 @@
import { assert } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { initSettingsFile, applyP2pSettings, applyP2pTestTweaks } from "./helpers/settings.ts";
import { startCliInBackground } from "./helpers/backgroundCli.ts";
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
Deno.test("p2p-peers: discovers host through local relay", async () => {
const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/";
const roomId = Deno.env.get("ROOM_ID") ?? `room-${Date.now()}`;
const passphrase = Deno.env.get("PASSPHRASE") ?? "test";
const timeoutSeconds = Number(Deno.env.get("TIMEOUT_SECONDS") ?? "8");
await using workDir = await TempDir.create("livesync-cli-p2p-peers-local-relay");
const hostVault = workDir.join("vault-host");
const hostSettings = workDir.join("settings-host.json");
const clientVault = workDir.join("vault");
const clientSettings = workDir.join("settings.json");
await Deno.mkdir(hostVault, { recursive: true });
await Deno.mkdir(clientVault, { recursive: true });
const relayStarted = await maybeStartLocalRelay(relay);
try {
await initSettingsFile(hostSettings);
await initSettingsFile(clientSettings);
await applyP2pSettings(hostSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
await applyP2pSettings(clientSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
await applyP2pTestTweaks(hostSettings, "p2p-host", passphrase);
await applyP2pTestTweaks(clientSettings, "p2p-client", passphrase);
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
try {
await host.waitUntilContains("P2P host is running", 20000);
const peer = await discoverPeer(clientVault, clientSettings, timeoutSeconds);
assert(peer.id.length > 0);
assert(peer.name.length > 0);
} finally {
await host.stop();
}
} finally {
await stopLocalRelayIfStarted(relayStarted);
}
});

View File

@@ -0,0 +1,59 @@
import { assert } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { initSettingsFile, applyP2pSettings, applyP2pTestTweaks } from "./helpers/settings.ts";
import { startCliInBackground } from "./helpers/backgroundCli.ts";
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
import { runCli } from "./helpers/cli.ts";
Deno.test("p2p-sync: discovers peer and completes sync", async () => {
const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/";
const roomId = Deno.env.get("ROOM_ID") ?? `room-${Date.now()}`;
const passphrase = Deno.env.get("PASSPHRASE") ?? "test";
const peersTimeout = Number(Deno.env.get("PEERS_TIMEOUT") ?? "12");
const syncTimeout = Number(Deno.env.get("SYNC_TIMEOUT") ?? "15");
await using workDir = await TempDir.create("livesync-cli-p2p-sync");
const hostVault = workDir.join("vault-host");
const hostSettings = workDir.join("settings-host.json");
const clientVault = workDir.join("vault-sync");
const clientSettings = workDir.join("settings-sync.json");
await Deno.mkdir(hostVault, { recursive: true });
await Deno.mkdir(clientVault, { recursive: true });
const relayStarted = await maybeStartLocalRelay(relay);
try {
await initSettingsFile(hostSettings);
await initSettingsFile(clientSettings);
await applyP2pSettings(hostSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
await applyP2pSettings(clientSettings, roomId, passphrase, "self-hosted-livesync-cli-tests", relay);
await applyP2pTestTweaks(hostSettings, "p2p-host", passphrase);
await applyP2pTestTweaks(clientSettings, "p2p-client", passphrase);
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
try {
await host.waitUntilContains("P2P host is running", 20000);
const peer = await discoverPeer(
clientVault,
clientSettings,
peersTimeout,
Deno.env.get("TARGET_PEER") ?? undefined
);
const syncResult = await runCli(
clientVault,
"--settings",
clientSettings,
"p2p-sync",
peer.id,
String(syncTimeout)
);
assert(
syncResult.code === 0,
`p2p-sync failed\nstdout: ${syncResult.stdout}\nstderr: ${syncResult.stderr}`
);
} finally {
await host.stop();
}
} finally {
await stopLocalRelayIfStarted(relayStarted);
}
});

View File

@@ -0,0 +1,118 @@
import { assert } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { applyP2pSettings, initSettingsFile } from "./helpers/settings.ts";
import { startCliInBackground } from "./helpers/backgroundCli.ts";
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
import { jsonStringField, runCliOrFail, runCliWithInputOrFail, sanitiseCatStdout } from "./helpers/cli.ts";
Deno.test("p2p: three nodes detect and resolve conflicts", async () => {
const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/";
const roomId = `${Deno.env.get("ROOM_ID_PREFIX") ?? "p2p-room"}-${Date.now()}`;
const passphrase = `${Deno.env.get("PASSPHRASE_PREFIX") ?? "p2p-pass"}-${Date.now()}`;
const appId = Deno.env.get("APP_ID") ?? "self-hosted-livesync-cli-tests";
const peersTimeout = Number(Deno.env.get("PEERS_TIMEOUT") ?? "10");
const syncTimeout = Number(Deno.env.get("SYNC_TIMEOUT") ?? "15");
await using workDir = await TempDir.create("livesync-cli-p2p-3nodes");
const vaultA = workDir.join("vault-a");
const vaultB = workDir.join("vault-b");
const vaultC = workDir.join("vault-c");
const settingsA = workDir.join("settings-a.json");
const settingsB = workDir.join("settings-b.json");
const settingsC = workDir.join("settings-c.json");
await Deno.mkdir(vaultA, { recursive: true });
await Deno.mkdir(vaultB, { recursive: true });
await Deno.mkdir(vaultC, { recursive: true });
const relayStarted = await maybeStartLocalRelay(relay);
try {
for (const settings of [settingsA, settingsB, settingsC]) {
await initSettingsFile(settings);
await applyP2pSettings(settings, roomId, passphrase, appId, relay);
}
const host = startCliInBackground(vaultA, "--settings", settingsA, "p2p-host");
try {
await host.waitUntilContains("P2P host is running", 20000);
const peerFromB = await discoverPeer(vaultB, settingsB, peersTimeout);
const peerFromC = await discoverPeer(vaultC, settingsC, peersTimeout);
const targetPath = "p2p/conflicted-from-two-clients.txt";
await runCliWithInputOrFail("from-client-b-v1\n", vaultB, "--settings", settingsB, "put", targetPath);
await runCliOrFail(vaultB, "--settings", settingsB, "p2p-sync", peerFromB.id, String(syncTimeout));
await runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout));
let visibleOnC = "";
for (let i = 0; i < 5; i++) {
try {
visibleOnC = sanitiseCatStdout(
await runCliOrFail(vaultC, "--settings", settingsC, "cat", targetPath)
).trimEnd();
if (visibleOnC === "from-client-b-v1") break;
} catch {
// retry below
}
await runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout));
}
assert(visibleOnC === "from-client-b-v1", `C should see file created by B, got: ${visibleOnC}`);
await runCliWithInputOrFail("from-client-b-v2\n", vaultB, "--settings", settingsB, "put", targetPath);
await runCliWithInputOrFail("from-client-c-v2\n", vaultC, "--settings", settingsC, "put", targetPath);
const [syncB, syncC] = await Promise.all([
runCliOrFail(vaultB, "--settings", settingsB, "p2p-sync", peerFromB.id, String(syncTimeout)),
runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout)),
]);
void syncB;
void syncC;
await runCliOrFail(vaultB, "--settings", settingsB, "p2p-sync", peerFromB.id, String(syncTimeout));
await runCliOrFail(vaultC, "--settings", settingsC, "p2p-sync", peerFromC.id, String(syncTimeout));
const infoBBefore = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetPath);
const conflictsBBefore = jsonStringField(infoBBefore, "conflicts");
const keepRevB = jsonStringField(infoBBefore, "revision");
assert(
conflictsBBefore !== "N/A" && conflictsBBefore.length > 0,
`expected conflicts on B\n${infoBBefore}`
);
assert(keepRevB.length > 0, `could not read revision on B\n${infoBBefore}`);
const infoCBefore = await runCliOrFail(vaultC, "--settings", settingsC, "info", targetPath);
const conflictsCBefore = jsonStringField(infoCBefore, "conflicts");
const keepRevC = jsonStringField(infoCBefore, "revision");
assert(
conflictsCBefore !== "N/A" && conflictsCBefore.length > 0,
`expected conflicts on C\n${infoCBefore}`
);
assert(keepRevC.length > 0, `could not read revision on C\n${infoCBefore}`);
await runCliOrFail(vaultB, "--settings", settingsB, "resolve", targetPath, keepRevB);
await runCliOrFail(vaultC, "--settings", settingsC, "resolve", targetPath, keepRevC);
const infoBAfter = await runCliOrFail(vaultB, "--settings", settingsB, "info", targetPath);
const infoCAfter = await runCliOrFail(vaultC, "--settings", settingsC, "info", targetPath);
assert(jsonStringField(infoBAfter, "conflicts") === "N/A", `conflict still remains on B\n${infoBAfter}`);
assert(jsonStringField(infoCAfter, "conflicts") === "N/A", `conflict still remains on C\n${infoCAfter}`);
const finalContentB = sanitiseCatStdout(
await runCliOrFail(vaultB, "--settings", settingsB, "cat", targetPath)
).trimEnd();
const finalContentC = sanitiseCatStdout(
await runCliOrFail(vaultC, "--settings", settingsC, "cat", targetPath)
).trimEnd();
assert(
finalContentB === "from-client-b-v2" || finalContentB === "from-client-c-v2",
`unexpected final content on B: ${finalContentB}`
);
assert(
finalContentC === "from-client-b-v2" || finalContentC === "from-client-c-v2",
`unexpected final content on C: ${finalContentC}`
);
} finally {
await host.stop();
}
} finally {
await stopLocalRelayIfStarted(relayStarted);
}
});

View File

@@ -0,0 +1,111 @@
import { TempDir } from "./helpers/temp.ts";
import { applyP2pSettings, applyP2pTestTweaks, initSettingsFile } from "./helpers/settings.ts";
import { startCliInBackground } from "./helpers/backgroundCli.ts";
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
async function writeFilledFile(path: string, size: number, byte: number): Promise<void> {
const data = new Uint8Array(size);
data.fill(byte);
await Deno.writeFile(path, data);
}
Deno.test("p2p: upload/download reproduction scenario", async () => {
const relay = Deno.env.get("RELAY") ?? "ws://localhost:4000/";
const appId = Deno.env.get("APP_ID") ?? "self-hosted-livesync-cli-tests";
const peersTimeout = Number(Deno.env.get("PEERS_TIMEOUT") ?? "20");
const syncTimeout = Number(Deno.env.get("SYNC_TIMEOUT") ?? "240");
const roomId = `p2p-room-${Date.now()}`;
const passphrase = `p2p-pass-${Date.now()}`;
await using workDir = await TempDir.create("livesync-cli-p2p-upload-download");
const vaultHost = workDir.join("vault-host");
const vaultUp = workDir.join("vault-up");
const vaultDown = workDir.join("vault-down");
const settingsHost = workDir.join("settings-host.json");
const settingsUp = workDir.join("settings-up.json");
const settingsDown = workDir.join("settings-down.json");
for (const dir of [vaultHost, vaultUp, vaultDown]) {
await Deno.mkdir(dir, { recursive: true });
}
const relayStarted = await maybeStartLocalRelay(relay);
try {
for (const settings of [settingsHost, settingsUp, settingsDown]) {
await initSettingsFile(settings);
await applyP2pSettings(settings, roomId, passphrase, appId, relay, "~.*");
}
await applyP2pTestTweaks(settingsHost, "p2p-cli-host", passphrase);
await applyP2pTestTweaks(settingsUp, `p2p-cli-upload-${Date.now()}`, passphrase);
await applyP2pTestTweaks(settingsDown, `p2p-cli-download-${Date.now()}`, passphrase);
const host = startCliInBackground(vaultHost, "--settings", settingsHost, "p2p-host");
try {
await host.waitUntilContains("P2P host is running", 20000);
const uploadPeer = await discoverPeer(vaultUp, settingsUp, peersTimeout);
const storeText = workDir.join("store-file.md");
const diffA = workDir.join("test-diff-1.md");
const diffB = workDir.join("test-diff-2.md");
const diffC = workDir.join("test-diff-3.md");
await Deno.writeTextFile(storeText, "Hello, World!\n");
await Deno.writeTextFile(diffA, "Content A\n");
await Deno.writeTextFile(diffB, "Content B\n");
await Deno.writeTextFile(diffC, "Content C\n");
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", storeText, "p2p/store-file.md");
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", diffA, "p2p/test-diff-1.md");
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", diffB, "p2p/test-diff-2.md");
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", diffC, "p2p/test-diff-3.md");
const large100k = workDir.join("large-100k.txt");
const large1m = workDir.join("large-1m.txt");
const binary100k = workDir.join("binary-100k.bin");
const binary5m = workDir.join("binary-5m.bin");
await Deno.writeTextFile(large100k, "a".repeat(100000));
await Deno.writeTextFile(large1m, "b".repeat(1000000));
await writeFilledFile(binary100k, 100000, 0x5a);
await writeFilledFile(binary5m, 5000000, 0x7c);
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", large100k, "p2p/large-100000.md");
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", large1m, "p2p/large-1000000.md");
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", binary100k, "p2p/binary-100000.bin");
await runCliOrFail(vaultUp, "--settings", settingsUp, "push", binary5m, "p2p/binary-5000000.bin");
await runCliOrFail(vaultUp, "--settings", settingsUp, "p2p-sync", uploadPeer.id, String(syncTimeout));
await runCliOrFail(vaultUp, "--settings", settingsUp, "p2p-sync", uploadPeer.id, String(syncTimeout));
const downloadPeer = await discoverPeer(vaultDown, settingsDown, peersTimeout);
await runCliOrFail(vaultDown, "--settings", settingsDown, "p2p-sync", downloadPeer.id, String(syncTimeout));
await runCliOrFail(vaultDown, "--settings", settingsDown, "p2p-sync", downloadPeer.id, String(syncTimeout));
const downStoreText = workDir.join("down-store-file.md");
const downDiffA = workDir.join("down-test-diff-1.md");
const downDiffB = workDir.join("down-test-diff-2.md");
const downDiffC = workDir.join("down-test-diff-3.md");
const downLarge100k = workDir.join("down-large-100k.txt");
const downLarge1m = workDir.join("down-large-1m.txt");
const downBinary100k = workDir.join("down-binary-100k.bin");
const downBinary5m = workDir.join("down-binary-5m.bin");
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/store-file.md", downStoreText);
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/test-diff-1.md", downDiffA);
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/test-diff-2.md", downDiffB);
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/test-diff-3.md", downDiffC);
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/large-100000.md", downLarge100k);
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/large-1000000.md", downLarge1m);
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/binary-100000.bin", downBinary100k);
await runCliOrFail(vaultDown, "--settings", settingsDown, "pull", "p2p/binary-5000000.bin", downBinary5m);
await assertFilesEqual(storeText, downStoreText, "store-file mismatch");
await assertFilesEqual(diffA, downDiffA, "test-diff-1 mismatch");
await assertFilesEqual(diffB, downDiffB, "test-diff-2 mismatch");
await assertFilesEqual(diffC, downDiffC, "test-diff-3 mismatch");
await assertFilesEqual(large100k, downLarge100k, "large-100000 mismatch");
await assertFilesEqual(large1m, downLarge1m, "large-1000000 mismatch");
await assertFilesEqual(binary100k, downBinary100k, "binary-100000 mismatch");
await assertFilesEqual(binary5m, downBinary5m, "binary-5000000 mismatch");
} finally {
await host.stop();
}
} finally {
await stopLocalRelayIfStarted(relayStarted);
}
});

View File

@@ -0,0 +1,78 @@
/**
* Deno port of test-push-pull-linux.sh
*
* Requires CouchDB connection details either via environment variables or a
* .test.env file. If neither is present the test logs a warning and the
* CLI will likely fail at the push step.
*
* Run:
* deno test -A test-push-pull.ts
*
* With explicit CouchDB:
* COUCHDB_URI=http://127.0.0.1:5984 \
* COUCHDB_USER=admin \
* COUCHDB_PASSWORD=password \
* COUCHDB_DBNAME=livesync-test \
* deno test -A test-push-pull.ts
*/
import { join } from "@std/path";
import { assertEquals } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { runCliOrFail } from "./helpers/cli.ts";
import { applyCouchdbSettings, initSettingsFile } from "./helpers/settings.ts";
import { startCouchdb, stopCouchdb } from "./helpers/docker.ts";
const REMOTE_PATH = Deno.env.get("REMOTE_PATH") ?? "test/push-pull.txt";
Deno.test("push/pull roundtrip", async () => {
await using workDir = await TempDir.create("livesync-cli-push-pull");
const settingsFile = workDir.join("data.json");
const vaultDir = workDir.join("vault");
await Deno.mkdir(join(vaultDir, "test"), { recursive: true });
const uri = Deno.env.get("COUCHDB_URI") ?? "http://127.0.0.1:5989/";
const user = Deno.env.get("COUCHDB_USER") ?? "admin";
const password = Deno.env.get("COUCHDB_PASSWORD") ?? "testpassword";
const dbname = Deno.env.get("COUCHDB_DBNAME") ?? `push-pull-${Date.now()}`;
const shouldStartDocker = Deno.env.get("LIVESYNC_START_DOCKER") !== "0";
const keepDocker = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1";
if (shouldStartDocker) {
await startCouchdb(uri, user, password, dbname);
}
try {
await initSettingsFile(settingsFile);
if (uri && user && password && dbname) {
console.log("[INFO] applying CouchDB env vars to settings");
await applyCouchdbSettings(settingsFile, uri, user, password, dbname);
} else {
console.warn(
"[WARN] CouchDB env vars not fully set — push/pull may fail unless the generated settings already contain connection details"
);
}
const srcFile = workDir.join("push-source.txt");
const pulledFile = workDir.join("pull-result.txt");
const content = `push-pull-test ${new Date().toISOString()}\n`;
await Deno.writeTextFile(srcFile, content);
console.log(`[INFO] push -> ${REMOTE_PATH}`);
await runCliOrFail(vaultDir, "--settings", settingsFile, "push", srcFile, REMOTE_PATH);
console.log(`[INFO] pull <- ${REMOTE_PATH}`);
await runCliOrFail(vaultDir, "--settings", settingsFile, "pull", REMOTE_PATH, pulledFile);
const pulled = await Deno.readTextFile(pulledFile);
assertEquals(content, pulled, "push/pull roundtrip content mismatch");
console.log("[PASS] push/pull roundtrip matched");
} finally {
if (shouldStartDocker && !keepDocker) {
await stopCouchdb().catch(() => {});
}
}
});

View File

@@ -0,0 +1,214 @@
/**
* Deno port of test-setup-put-cat-linux.sh
*
* Tests all local-DB file operations that require no external remote:
* setup /
* push / cat / ls / info / rm / resolve / cat-rev / pull-rev
*
* Run (no external services needed):
* deno test -A test-setup-put-cat.ts
*/
import { join } from "@std/path";
import { assertEquals, assert } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { runCli, runCliOrFail, runCliWithInput, sanitiseCatStdout } from "./helpers/cli.ts";
import { generateSetupUriFromSettings, initSettingsFile } from "./helpers/settings.ts";
const REMOTE_PATH = Deno.env.get("REMOTE_PATH") ?? "test/setup-put-cat.txt";
const SETUP_PASSPHRASE = Deno.env.get("SETUP_PASSPHRASE") ?? "setup-passphrase";
Deno.test("CLI file operations: push / cat / ls / info / rm / resolve / cat-rev / pull-rev", async (t) => {
await using workDir = await TempDir.create("livesync-cli-setup-put-cat");
const settingsFile = workDir.join("data.json");
const vaultDir = workDir.join("vault");
await Deno.mkdir(join(vaultDir, "test"), { recursive: true });
await initSettingsFile(settingsFile);
const setupUri = await generateSetupUriFromSettings(settingsFile, SETUP_PASSPHRASE);
const setupResult = await runCliWithInput(
`${SETUP_PASSPHRASE}\n`,
vaultDir,
"--settings",
settingsFile,
"setup",
setupUri
);
assert(setupResult.code === 0, `setup command exited with ${setupResult.code}\n${setupResult.combined}`);
assert(
setupResult.combined.includes("[Command] setup ->"),
`setup command did not execute expected code path\n${setupResult.combined}`
);
const run = (...args: string[]) => runCliOrFail(vaultDir, "--settings", settingsFile, ...args);
// ------------------------------------------------------------------
// push / cat roundtrip
// ------------------------------------------------------------------
await t.step("push/cat roundtrip", async () => {
const srcFile = workDir.join("put-source.txt");
const content = `setup-put-cat-test ${new Date().toISOString()}\nline-2\n`;
await Deno.writeTextFile(srcFile, content);
console.log(`[INFO] push -> ${REMOTE_PATH}`);
await runCliWithInput(content, vaultDir, "--settings", settingsFile, "put", REMOTE_PATH);
console.log(`[INFO] cat <- ${REMOTE_PATH}`);
const rawOutput = await run("cat", REMOTE_PATH);
const catOutput = sanitiseCatStdout(rawOutput);
assertEquals(content, catOutput, "push/cat roundtrip content mismatch");
console.log("[PASS] push/cat roundtrip matched");
});
// ------------------------------------------------------------------
// ls: single file
// ------------------------------------------------------------------
await t.step("ls output format (single file)", async () => {
const lsOutput = await run("ls", REMOTE_PATH);
const line = lsOutput
.trim()
.split("\n")
.find((l) => l.startsWith(REMOTE_PATH + "\t"));
assert(line, `ls output did not include ${REMOTE_PATH}`);
const [lsPath, lsSize, lsMtime, lsRev] = line.split("\t");
assertEquals(lsPath, REMOTE_PATH, "ls path column mismatch");
assert(/^\d+$/.test(lsSize), `ls size not numeric: ${lsSize}`);
assert(/^\d+$/.test(lsMtime), `ls mtime not numeric: ${lsMtime}`);
assert(lsRev?.length > 0, "ls revision column is empty");
console.log("[PASS] ls output format matched");
});
// ------------------------------------------------------------------
// ls: prefix filter and sort order
// ------------------------------------------------------------------
await t.step("ls prefix filter and sort order", async () => {
await runCliWithInput("file-a\n", vaultDir, "--settings", settingsFile, "put", "test/a-first.txt");
await runCliWithInput("file-z\n", vaultDir, "--settings", settingsFile, "put", "test/z-last.txt");
const lsOut = await run("ls", "test/");
const lines = lsOut.trim().split("\n").filter(Boolean);
assert(lines.length >= 3, "ls prefix output expected at least 3 rows");
// Verify sorted ascending by path
const paths = lines.map((l) => l.split("\t")[0]);
for (let i = 1; i < paths.length; i++) {
assert(paths[i - 1] <= paths[i], `ls output not sorted: ${paths[i - 1]} > ${paths[i]}`);
}
assert(
lines.some((l) => l.startsWith("test/a-first.txt\t")),
"ls prefix output missing test/a-first.txt"
);
assert(
lines.some((l) => l.startsWith("test/z-last.txt\t")),
"ls prefix output missing test/z-last.txt"
);
console.log("[PASS] ls prefix and sorting matched");
});
// ------------------------------------------------------------------
// ls: no-match prefix returns empty output
// ------------------------------------------------------------------
await t.step("ls no-match prefix returns empty", async () => {
const lsOut = await run("ls", "no-such-prefix/");
assertEquals(lsOut.trim(), "", "ls no-match prefix should produce empty output");
console.log("[PASS] ls no-match prefix matched");
});
// ------------------------------------------------------------------
// info: JSON output format
// ------------------------------------------------------------------
await t.step("info output JSON format", async () => {
const infoOut = await run("info", REMOTE_PATH);
let data: Record<string, unknown>;
try {
data = JSON.parse(infoOut);
} catch {
throw new Error(`info output is not valid JSON:\n${infoOut}`);
}
assertEquals(data.path, REMOTE_PATH, "info .path mismatch");
assertEquals(data.filename, REMOTE_PATH.split("/").at(-1), "info .filename mismatch");
assert(typeof data.size === "number" && data.size >= 0, `info .size invalid: ${data.size}`);
assert(typeof data.chunks === "number" && (data.chunks as number) >= 1, `info .chunks invalid: ${data.chunks}`);
assertEquals(data.conflicts, "N/A", "info .conflicts should be N/A");
console.log("[PASS] info output format matched");
});
// ------------------------------------------------------------------
// info: non-existent path exits non-zero
// ------------------------------------------------------------------
await t.step("info non-existent path returns non-zero", async () => {
const r = await runCli(vaultDir, "--settings", settingsFile, "info", "no-such-file.md");
assert(r.code !== 0, "info on non-existent file should exit non-zero");
console.log("[PASS] info non-existent path returns non-zero");
});
// ------------------------------------------------------------------
// rm: removes file from ls and makes cat fail
// ------------------------------------------------------------------
await t.step("rm removes target from ls and cat", async () => {
await run("rm", "test/z-last.txt");
const catResult = await runCli(vaultDir, "--settings", settingsFile, "cat", "test/z-last.txt");
assert(catResult.code !== 0, "rm target should not be readable by cat");
const lsOut = await run("ls", "test/");
assert(!lsOut.includes("test/z-last.txt\t"), "rm target should not appear in ls output");
console.log("[PASS] rm removed target from visible entries");
});
// ------------------------------------------------------------------
// resolve: accepts current revision, rejects invalid revision
// ------------------------------------------------------------------
await t.step("resolve: valid and invalid revisions", async () => {
const lsLine = (await run("ls", "test/a-first.txt")).trim().split("\n")[0];
assert(lsLine, "could not fetch revision for resolve test");
const rev = lsLine.split("\t")[3];
assert(rev?.length > 0, "revision was empty for resolve test");
await run("resolve", "test/a-first.txt", rev);
console.log("[PASS] resolve accepted current revision");
const badR = await runCli(vaultDir, "--settings", settingsFile, "resolve", "test/a-first.txt", "9-no-such-rev");
assert(badR.code !== 0, "resolve with non-existent revision should exit non-zero");
console.log("[PASS] resolve non-existent revision returns non-zero");
});
// ------------------------------------------------------------------
// cat-rev / pull-rev: retrieve a past revision
// ------------------------------------------------------------------
await t.step("cat-rev / pull-rev: retrieve past revision", async () => {
const revPath = "test/revision-history.txt";
await runCliWithInput("revision-v1\n", vaultDir, "--settings", settingsFile, "put", revPath);
await runCliWithInput("revision-v2\n", vaultDir, "--settings", settingsFile, "put", revPath);
await runCliWithInput("revision-v3\n", vaultDir, "--settings", settingsFile, "put", revPath);
const infoOut = await run("info", revPath);
const infoData = JSON.parse(infoOut) as {
revisions?: string[];
};
const revisions = Array.isArray(infoData.revisions) ? infoData.revisions : [];
const pastRev = revisions.find((r): r is string => typeof r === "string" && r !== "N/A");
assert(pastRev, "info output did not include any past revision");
const catRevOut = await run("cat-rev", revPath, pastRev);
const catRevClean = sanitiseCatStdout(catRevOut);
assert(
catRevClean === "revision-v1\n" || catRevClean === "revision-v2\n",
`cat-rev output did not match expected past revision:\n${catRevClean}`
);
console.log("[PASS] cat-rev matched one of the past revisions from info");
const pullRevFile = workDir.join("rev-pull-output.txt");
await run("pull-rev", revPath, pullRevFile, pastRev);
const pullRevContent = await Deno.readTextFile(pullRevFile);
assert(
pullRevContent === "revision-v1\n" || pullRevContent === "revision-v2\n",
`pull-rev output did not match expected past revision:\n${pullRevContent}`
);
console.log("[PASS] pull-rev matched one of the past revisions from info");
});
});

View File

@@ -0,0 +1,93 @@
/**
* Deno port of test-sync-locked-remote-linux.sh
*
* Verifies CLI sync behaviour when the remote milestone document is unlocked
* versus locked.
*/
import { assert, assertStringIncludes } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { runCli } from "./helpers/cli.ts";
import { applyCouchdbSettings, initSettingsFile } from "./helpers/settings.ts";
import { createCouchdbDatabase, startCouchdb, stopCouchdb, updateCouchdbDoc } from "./helpers/docker.ts";
const MILESTONE_DOC = "_local/obsydian_livesync_milestone";
function requireEnv(...keys: string[]): string {
for (const key of keys) {
const value = Deno.env.get(key)?.trim();
if (value) return value;
}
throw new Error(`Required env var is missing: ${keys.join(" or ")}`);
}
Deno.test("sync: actionable error against locked remote DB", async () => {
const couchdbUri = requireEnv("COUCHDB_URI", "hostname").replace(/\/$/, "");
const couchdbUser = requireEnv("COUCHDB_USER", "username");
const couchdbPassword = requireEnv("COUCHDB_PASSWORD", "password");
const dbPrefix = requireEnv("COUCHDB_DBNAME", "dbname");
const dbname = `${dbPrefix}-locked-${Date.now()}-${Math.floor(Math.random() * 100000)}`;
await using workDir = await TempDir.create("livesync-cli-locked-test");
const vaultDir = workDir.join("vault");
const settingsFile = workDir.join("settings.json");
await Deno.mkdir(vaultDir, { recursive: true });
const shouldStartDocker = Deno.env.get("LIVESYNC_START_DOCKER") !== "0";
const keepDocker = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1";
if (shouldStartDocker) {
console.log(`[INFO] starting CouchDB and creating test database: ${dbname}`);
await startCouchdb(couchdbUri, couchdbUser, couchdbPassword, dbname);
} else {
console.log(`[INFO] using existing CouchDB and creating test database: ${dbname}`);
await createCouchdbDatabase(couchdbUri, couchdbUser, couchdbPassword, dbname);
}
try {
await initSettingsFile(settingsFile);
await applyCouchdbSettings(settingsFile, couchdbUri, couchdbUser, couchdbPassword, dbname, true);
console.log("[CASE] initial sync to create milestone document");
const initialSync = await runCli(vaultDir, "--settings", settingsFile, "sync");
assert(
initialSync.code === 0,
`initial sync failed\nstdout: ${initialSync.stdout}\nstderr: ${initialSync.stderr}`
);
const updateMilestone = async (locked: boolean) => {
await updateCouchdbDoc(couchdbUri, couchdbUser, couchdbPassword, `${dbname}/${MILESTONE_DOC}`, (doc) => ({
...doc,
locked,
accepted_nodes: [],
}));
};
console.log("[CASE] sync should succeed when remote is not locked");
await updateMilestone(false);
const unlockedSync = await runCli(vaultDir, "--settings", settingsFile, "sync");
assert(
unlockedSync.code === 0,
`sync should succeed when remote is not locked\nstdout: ${unlockedSync.stdout}\nstderr: ${unlockedSync.stderr}`
);
assert(
!unlockedSync.combined.includes("The remote database is locked"),
`locked error should not appear when remote is not locked\n${unlockedSync.combined}`
);
console.log("[PASS] unlocked remote DB syncs successfully");
console.log("[CASE] sync should fail with actionable error when remote is locked");
await updateMilestone(true);
const lockedSync = await runCli(vaultDir, "--settings", settingsFile, "sync");
assert(
lockedSync.code !== 0,
`sync should fail when remote is locked\nstdout: ${lockedSync.stdout}\nstderr: ${lockedSync.stderr}`
);
assertStringIncludes(lockedSync.combined, "The remote database is locked and this device is not yet accepted");
console.log("[PASS] locked remote DB produces actionable CLI error");
} finally {
if (shouldStartDocker && !keepDocker) {
await stopCouchdb().catch(() => {});
}
}
});

View File

@@ -0,0 +1,272 @@
/**
* Deno port of test-sync-two-local-databases-linux.sh
*
* Tests two-vault synchronisation via CouchDB including conflict detection
* and resolution.
*
* Requires CouchDB connection details. Provide them via environment variables
* OR place a .test.env file at src/apps/cli/.test.env.
*
* By default, a CouchDB Docker container is started automatically
* (LIVESYNC_START_DOCKER=1). Set LIVESYNC_START_DOCKER=0 to use an existing
* CouchDB instance instead.
*
* Run:
* deno test -A test-sync-two-local-databases.ts
*
* With an existing CouchDB:
* COUCHDB_URI=http://127.0.0.1:5984 \
* COUCHDB_USER=admin \
* COUCHDB_PASSWORD=password \
* COUCHDB_DBNAME=livesync-test \
* LIVESYNC_START_DOCKER=0 \
* deno test -A test-sync-two-local-databases.ts
*/
import { assertEquals, assert } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { runCliOrFail, jsonFieldIsNa } from "./helpers/cli.ts";
import { applyCouchdbSettings, initSettingsFile } from "./helpers/settings.ts";
import { startCouchdb, stopCouchdb } from "./helpers/docker.ts";
// ---------------------------------------------------------------------------
// Load configuration
// ---------------------------------------------------------------------------
async function resolveConfig(): Promise<{
uri: string;
user: string;
password: string;
baseDbname: string;
} | null> {
const env = Deno.env.toObject();
const uri = (env["COUCHDB_URI"] ?? env["hostname"] ?? "").replace(/\/$/, "");
const user = env["COUCHDB_USER"] ?? env["username"] ?? "";
const password = env["COUCHDB_PASSWORD"] ?? env["password"] ?? "";
const baseDbname = env["COUCHDB_DBNAME"] ?? env["dbname"] ?? "livesync-test";
if (!uri || !user || !password) return null;
return { uri, user, password, baseDbname };
}
const config = await resolveConfig();
const START_DOCKER = Deno.env.get("LIVESYNC_START_DOCKER") !== "0";
const KEEP_DOCKER = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1";
const SYNC_RETRY = Number(Deno.env.get("LIVESYNC_SYNC_RETRY") ?? "8");
// Provide a sane default for flaky remote connectivity in Docker-on-WSL
// environments. Users can override explicitly if needed.
if (!Deno.env.has("LIVESYNC_CLI_RETRY")) {
Deno.env.set("LIVESYNC_CLI_RETRY", "2");
}
// ---------------------------------------------------------------------------
// Test suite
// ---------------------------------------------------------------------------
Deno.test(
{
name: "sync two local databases: sync + conflict detection + resolution",
ignore: config === null,
},
async (t) => {
if (!config) return; // narrowing for TypeScript
const suffix = `${Date.now()}-${Math.floor(Math.random() * 65535)}`;
const dbname = `${config.baseDbname}-${suffix}`;
await using workDir = await TempDir.create("livesync-cli-two-db-test");
// ------------------------------------------------------------------
// Docker lifecycle
// ------------------------------------------------------------------
if (START_DOCKER) {
await startCouchdb(config.uri, config.user, config.password, dbname);
}
try {
await runSuite(t, workDir, config, dbname);
} finally {
if (START_DOCKER && !KEEP_DOCKER) {
await stopCouchdb().catch(() => {});
}
if (START_DOCKER && KEEP_DOCKER) {
console.log("[INFO] LIVESYNC_DEBUG_KEEP_DOCKER=1, keeping couchdb-test container");
}
console.log(`[INFO] test database '${dbname}' is preserved for debugging.`);
}
}
);
// ---------------------------------------------------------------------------
// Suite implementation
// ---------------------------------------------------------------------------
async function runSuite(
t: Deno.TestContext,
workDir: TempDir,
config: { uri: string; user: string; password: string },
dbname: string
): Promise<void> {
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
const runWithRetry = async <T>(label: string, fn: () => Promise<T>, retries = SYNC_RETRY): Promise<T> => {
let lastErr: unknown;
for (let i = 0; i <= retries; i++) {
try {
return await fn();
} catch (err) {
lastErr = err;
if (i === retries) break;
const delayMs = 500 * (i + 1);
console.warn(`[WARN] ${label} failed, retrying (${i + 1}/${retries}) in ${delayMs}ms`);
await sleep(delayMs);
}
}
throw lastErr;
};
const vaultA = workDir.join("vault-a");
const vaultB = workDir.join("vault-b");
const settingsA = workDir.join("a-settings.json");
const settingsB = workDir.join("b-settings.json");
await Deno.mkdir(vaultA, { recursive: true });
await Deno.mkdir(vaultB, { recursive: true });
await initSettingsFile(settingsA);
await initSettingsFile(settingsB);
const applySettings = async (f: string) =>
applyCouchdbSettings(f, config.uri, config.user, config.password, dbname, /* liveSync */ true);
await applySettings(settingsA);
await applySettings(settingsB);
const runA = (...args: string[]) => runCliOrFail(vaultA, "--settings", settingsA, ...args);
const runB = (...args: string[]) => runCliOrFail(vaultB, "--settings", settingsB, ...args);
const syncA = () => runWithRetry("syncA", () => runA("sync"));
const syncB = () => runWithRetry("syncB", () => runB("sync"));
const catA = (path: string) => runA("cat", path);
const catB = (path: string) => runB("cat", path);
// ------------------------------------------------------------------
// Case 1: A creates file, B reads after sync
// ------------------------------------------------------------------
await t.step("case 1: A creates file -> B can read after sync", async () => {
const srcA = workDir.join("from-a-src.txt");
await Deno.writeTextFile(srcA, "from-a\n");
await runA("push", srcA, "shared/from-a.txt");
await syncA();
await syncB();
const value = (await catB("shared/from-a.txt")).replace(/\r\n/g, "\n").trimEnd();
assertEquals(value, "from-a", "B could not read file created on A");
console.log("[PASS] case 1 passed");
});
// ------------------------------------------------------------------
// Case 2: B creates file, A reads after sync
// ------------------------------------------------------------------
await t.step("case 2: B creates file -> A can read after sync", async () => {
const srcB = workDir.join("from-b-src.txt");
await Deno.writeTextFile(srcB, "from-b\n");
await runB("push", srcB, "shared/from-b.txt");
await syncB();
await syncA();
const value = (await catA("shared/from-b.txt")).replace(/\r\n/g, "\n").trimEnd();
assertEquals(value, "from-b", "A could not read file created on B");
console.log("[PASS] case 2 passed");
});
// ------------------------------------------------------------------
// Case 3: concurrent edits create a conflict
// ------------------------------------------------------------------
await t.step("case 3: concurrent edits create conflict", async () => {
const baseSrc = workDir.join("base-src.txt");
await Deno.writeTextFile(baseSrc, "base\n");
await runA("push", baseSrc, "shared/conflicted.txt");
await syncA();
await syncB();
const aEdit = workDir.join("edit-a.txt");
const bEdit = workDir.join("edit-b.txt");
await Deno.writeTextFile(aEdit, "edit-from-a\n");
await Deno.writeTextFile(bEdit, "edit-from-b\n");
await runA("push", aEdit, "shared/conflicted.txt");
await runB("push", bEdit, "shared/conflicted.txt");
const infoFileA = workDir.join("info-a.json");
const infoFileB = workDir.join("info-b.json");
let conflictDetected = false;
for (const side of ["a", "b"] as const) {
if (side === "a") await syncA();
else await syncB();
await Deno.writeTextFile(infoFileA, await runA("info", "shared/conflicted.txt"));
await Deno.writeTextFile(infoFileB, await runB("info", "shared/conflicted.txt"));
const da = JSON.parse(await Deno.readTextFile(infoFileA)) as Record<string, unknown>;
const db = JSON.parse(await Deno.readTextFile(infoFileB)) as Record<string, unknown>;
if (!jsonFieldIsNa(da, "conflicts") || !jsonFieldIsNa(db, "conflicts")) {
conflictDetected = true;
break;
}
}
assert(conflictDetected, "expected conflict after concurrent edits, but both sides show N/A");
console.log("[PASS] case 3 conflict detected");
});
// ------------------------------------------------------------------
// Case 4: resolve on A, verify B has no conflict after sync
// ------------------------------------------------------------------
await t.step("case 4: resolve on A propagates to B", async () => {
const infoFileA = workDir.join("info-a-resolve.json");
const infoFileB = workDir.join("info-b-resolve.json");
// Ensure A sees the conflict
for (let i = 0; i < 5; i++) {
const raw = await runA("info", "shared/conflicted.txt");
await Deno.writeTextFile(infoFileA, raw);
const da = JSON.parse(raw) as Record<string, unknown>;
if (!jsonFieldIsNa(da, "conflicts")) break;
await syncB();
await syncA();
}
const rawA = await runA("info", "shared/conflicted.txt");
await Deno.writeTextFile(infoFileA, rawA);
const dataA = JSON.parse(rawA) as Record<string, unknown>;
assert(!jsonFieldIsNa(dataA, "conflicts"), "A does not see conflict, cannot resolve from A only");
const keepRev = dataA["revision"] as string;
assert(keepRev?.length > 0, "could not read revision from A info output");
await runA("resolve", "shared/conflicted.txt", keepRev);
let resolved = false;
for (let i = 0; i < 6; i++) {
await syncA();
await syncB();
const rawA2 = await runA("info", "shared/conflicted.txt");
const rawB2 = await runB("info", "shared/conflicted.txt");
await Deno.writeTextFile(infoFileA, rawA2);
await Deno.writeTextFile(infoFileB, rawB2);
const da2 = JSON.parse(rawA2) as Record<string, unknown>;
const db2 = JSON.parse(rawB2) as Record<string, unknown>;
if (jsonFieldIsNa(da2, "conflicts") && jsonFieldIsNa(db2, "conflicts")) {
resolved = true;
break;
}
// If A still sees a conflict, resolve it again
if (!jsonFieldIsNa(da2, "conflicts")) {
const rev2 = da2["revision"] as string;
if (rev2) await runA("resolve", "shared/conflicted.txt", rev2).catch(() => {});
}
}
assert(resolved, "conflicts should be resolved on both A and B");
const contentA = (await catA("shared/conflicted.txt")).replace(/\r\n/g, "\n");
const contentB = (await catB("shared/conflicted.txt")).replace(/\r\n/g, "\n");
assertEquals(contentA, contentB, "resolved content mismatch between A and B");
console.log("[PASS] case 4 passed");
console.log("[PASS] all sync/resolve scenarios passed");
});
}

View File

@@ -0,0 +1,298 @@
# CLI Deno Test Development Notes
This document provides an overview of the Deno-based compatibility tests under `src/apps/cli/testdeno/`.
The existing bash tests under `src/apps/cli/test/` are preserved, while a Windows-friendly suite is maintained in parallel.
---
## Goals
- Keep existing bash tests intact.
- Provide direct execution from Windows PowerShell.
- Establish a TypeScript (Deno) foundation for core end-to-end and integration scenarios.
---
## Directory structure
```
src/apps/cli/testdeno/
deno.json
CONTRIBUTING_TESTS.md
helpers/
backgroundCli.ts
cli.ts
docker.ts
env.ts
p2p.ts
settings.ts
temp.ts
test-e2e-two-vaults-couchdb.ts
test-push-pull.ts
test-p2p-host.ts
test-p2p-peers-local-relay.ts
test-p2p-sync.ts
test-p2p-three-nodes-conflict.ts
test-p2p-upload-download-repro.ts
test-e2e-two-vaults-matrix.ts
test-setup-put-cat.ts
test-mirror.ts
test-sync-two-local-databases.ts
test-sync-locked-remote.ts
```
---
## Key files
### `deno.json`
- Defines Deno tasks.
- Defines import maps for `@std/assert` and `@std/path`.
Main tasks:
- `deno task test`
- `deno task test:local`
- `deno task test:push-pull`
- `deno task test:setup-put-cat`
- `deno task test:mirror`
- `deno task test:sync-two-local`
- `deno task test:sync-locked-remote`
- `deno task test:p2p-host`
- `deno task test:p2p-peers`
- `deno task test:p2p-sync`
- `deno task test:p2p-three-nodes`
- `deno task test:p2p-upload-download`
- `deno task test:e2e-couchdb`
- `deno task test:e2e-matrix`
### `helpers/cli.ts`
- CLI execution wrappers.
- `runCli`, `runCliOrFail`, `runCliWithInput`.
- Output normalisation via `sanitiseCatStdout`.
- Comparison utilities, including `assertFilesEqual`.
This file corresponds to `run_cli` and common assertions in `test-helpers.sh`.
### `helpers/settings.ts`
- Executes `init-settings --force`.
- Marks `isConfigured = true`.
- Applies CouchDB and P2P settings.
- Applies remote synchronisation settings and P2P test tweaks.
This file corresponds to settings helpers in `test-helpers.sh`.
### `helpers/docker.ts`
- Starts, stops, and initialises CouchDB directly from Deno.
- Configures CouchDB via `fetch + retry`.
- Starts and stops the P2P relay through the same Docker runner.
Both CouchDB and P2P relay flows are bash-independent.
### `helpers/backgroundCli.ts`
- Starts long-running commands such as `p2p-host` in the background.
- Waits for readiness logs and handles termination.
### `helpers/p2p.ts`
- Determines whether a local relay should be started.
- Parses `p2p-peers` output.
- Discovers peer IDs with a fallback based on advertisement logs.
### `helpers/env.ts`
- Loads `.test.env`.
- Supports `KEY=value`, single-quoted values, and double-quoted values.
### `helpers/temp.ts`
- Provides `TempDir`.
- Uses `await using` to auto-clean temporary directories.
---
## Implemented tests
### `test-push-pull.ts`
- Verifies push and pull round trips.
- Uses environment variables or `.test.env` for CouchDB values.
### `test-setup-put-cat.ts`
- Verifies `setup` with full setup URI generation via `encodeSettingsToSetupURI`.
- Verifies `push`, `cat`, `ls`, `info`, `rm`, `resolve`, `cat-rev`, and `pull-rev`.
- Does not require an external remote.
### `test-mirror.ts`
- Verifies six core mirror scenarios.
- Does not require an external remote.
### `test-sync-two-local-databases.ts`
- Verifies sync between two vaults and CouchDB.
- Verifies conflict detection and resolve propagation.
- Starts Docker CouchDB by default when `LIVESYNC_START_DOCKER != 0`.
### `test-sync-locked-remote.ts`
- Updates the CouchDB milestone `locked` flag.
- Verifies sync success when unlocked.
- Verifies actionable CLI error when locked.
### `test-p2p-host.ts`
- Verifies that `p2p-host` starts and emits readiness output.
### `test-p2p-peers-local-relay.ts`
- Verifies peer discovery through a local relay.
### `test-p2p-sync.ts`
- Verifies that `p2p-sync` completes after peer discovery.
### `test-p2p-three-nodes-conflict.ts`
- Uses one host and two clients.
- Verifies conflict creation, detection via `info`, and resolution via `resolve`.
### `test-p2p-upload-download-repro.ts`
- Uses host, upload, and download nodes.
- Verifies transfer of text files and binary files, including larger files.
### `test-e2e-two-vaults-couchdb.ts`
- Verifies two-vault end-to-end scenarios on CouchDB.
- Runs both encryption-off and encryption-on cases.
- Includes conflict marker checks in `ls` and resolve propagation checks.
### `test-e2e-two-vaults-matrix.ts`
- Verifies the matrix equivalent of the bash script.
- Runs four combinations:
- `COUCHDB-enc0`
- `COUCHDB-enc1`
- `MINIO-enc0`
- `MINIO-enc1`
---
## Running tests (PowerShell)
From `src/apps/cli/testdeno`:
```powershell
cd src/apps/cli/testdeno
# Local-only set
deno task test:local
# Individual tests
deno task test:setup-put-cat
deno task test:mirror
deno task test:push-pull
deno task test:sync-locked-remote
# CouchDB-based tests
deno task test:sync-two-local
deno task test:e2e-couchdb
# P2P-based tests
deno task test:p2p-host
deno task test:p2p-peers
deno task test:p2p-sync
deno task test:p2p-three-nodes
deno task test:p2p-upload-download
deno task test:e2e-matrix
```
---
## Environment variables
### CouchDB
- `COUCHDB_URI`
- `COUCHDB_USER`
- `COUCHDB_PASSWORD`
- `COUCHDB_DBNAME`
Equivalent keys in `src/apps/cli/.test.env`:
- `hostname`
- `username`
- `password`
- `dbname`
### Behaviour switches
- `LIVESYNC_START_DOCKER=0`: use existing CouchDB.
- `REMOTE_PATH`: override target path for selected tests.
- `LIVESYNC_TEST_TEE=1`: stream CLI stdout and stderr during execution.
- `LIVESYNC_DOCKER_TEE=1`: stream Docker stdout and stderr.
- `LIVESYNC_CLI_RETRY=<n>`: retry transient network failures.
- `LIVESYNC_DEBUG_KEEP_DOCKER=1`: keep `couchdb-test` after test completion.
### Docker command selection
`helpers/docker.ts` supports command selection via environment variables.
- `LIVESYNC_DOCKER_MODE=auto` (default)
- Windows: tries `wsl docker` first, then `docker`.
- Non-Windows: tries `docker` first, then `wsl docker`.
- `LIVESYNC_DOCKER_MODE=native`: always uses `docker`.
- `LIVESYNC_DOCKER_MODE=wsl`: always uses `wsl docker`.
- `LIVESYNC_DOCKER_COMMAND="..."`: custom command, for example `wsl docker`.
`LIVESYNC_DOCKER_COMMAND` has priority over `LIVESYNC_DOCKER_MODE`.
PowerShell examples:
```powershell
# Use Docker in WSL explicitly
$env:LIVESYNC_DOCKER_MODE = "wsl"
deno task test:sync-two-local
# Full custom command
$env:LIVESYNC_DOCKER_COMMAND = "wsl docker"
deno task test:sync-two-local
```
### P2P
- `RELAY`
- `ROOM_ID`
- `PASSPHRASE`
- `APP_ID`
- `PEERS_TIMEOUT`
- `SYNC_TIMEOUT`
- `USE_INTERNAL_RELAY=0|1`
- `TIMEOUT_SECONDS`
---
## Continuous Integration
The GitHub Actions workflow `.github/workflows/cli-deno-tests.yml` is used to run these tests automatically on push and pull requests affecting the CLI.
---
## Current limitations
- MinIO startup and matrix coverage are ported. Current limits are elsewhere, not setup URI generation.
---
## Maintenance policy
- Existing bash tests remain available.
- Deno tests are expanded in parallel for cross-platform usage.
- New scenarios should be added through reusable helpers in `helpers/`.

View File

@@ -2,6 +2,7 @@ import { defineConfig } from "vite";
import { svelte } from "@sveltejs/vite-plugin-svelte";
import path from "node:path";
import { readFileSync } from "node:fs";
const resolve = (...args: string[]) => path.resolve(...args).replace(/\\/g, "/");
const packageJson = JSON.parse(readFileSync("../../../package.json", "utf-8"));
const manifestJson = JSON.parse(readFileSync("../../../manifest.json", "utf-8"));
// https://vite.dev/config/
@@ -11,25 +12,66 @@ const defaultExternal = [
"crypto",
"pouchdb-adapter-leveldb",
"commander",
"chokidar",
"punycode",
"werift",
];
// Polyfill FileReader at the very top of the CJS bundle. octagonal-wheels uses
// FileReader for base64 conversion when Uint8Array.toBase64 (TC39 proposal) is
// unavailable. Node.js has neither, so we inject a minimal FileReader shim before
// any module-scope code evaluates.
const fileReaderPolyfillBanner = `
if (typeof globalThis.FileReader === "undefined") {
globalThis.FileReader = class FileReader {
constructor() { this.result = null; this.onload = null; this.onerror = null; }
readAsDataURL(blob) {
blob.arrayBuffer().then((buf) => {
var b64 = require("buffer").Buffer.from(buf).toString("base64");
this.result = "data:" + (blob.type || "application/octet-stream") + ";base64," + b64;
if (this.onload) this.onload({ target: this });
}).catch((err) => { if (this.onerror) this.onerror({ target: this, error: err }); });
}
readAsArrayBuffer() { throw new Error("FileReader.readAsArrayBuffer is not implemented in this polyfill"); }
readAsBinaryString() { throw new Error("FileReader.readAsBinaryString is not implemented in this polyfill"); }
readAsText() { throw new Error("FileReader.readAsText is not implemented in this polyfill"); }
abort() { throw new Error("FileReader.abort is not implemented in this polyfill"); }
};
}
`;
function injectBanner(): import("vite").Plugin {
return {
name: "inject-banner",
generateBundle(_options, bundle) {
for (const chunk of Object.values(bundle)) {
if (chunk.type === "chunk" && chunk.fileName.startsWith("entrypoint")) {
// Insert after the shebang line if present, otherwise at the top.
if (chunk.code.startsWith("#!")) {
const newline = chunk.code.indexOf("\n");
chunk.code =
chunk.code.slice(0, newline + 1) + fileReaderPolyfillBanner + chunk.code.slice(newline + 1);
} else {
chunk.code = fileReaderPolyfillBanner + chunk.code;
}
}
}
},
};
}
export default defineConfig({
plugins: [svelte()],
plugins: [svelte(), injectBanner()],
resolve: {
alias: {
"@lib/worker/bgWorker.ts": "../../lib/src/worker/bgWorker.mock.ts",
"@lib/pouchdb/pouchdb-browser.ts": path.resolve(__dirname, "lib/pouchdb-node.ts"),
"@lib/pouchdb/pouchdb-browser.ts": resolve(__dirname, "lib/pouchdb-node.ts"),
// The CLI runs on Node.js; force AWS XML builder to its CJS Node entry
// so Vite does not resolve the browser DOMParser-based XML parser.
"@aws-sdk/xml-builder": path.resolve(
__dirname,
"../../../node_modules/@aws-sdk/xml-builder/dist-cjs/index.js"
),
"@aws-sdk/xml-builder": resolve(__dirname, "../../../node_modules/@aws-sdk/xml-builder/dist-cjs/index.js"),
// Force fflate to the Node CJS entry; browser entry expects Web Worker globals.
fflate: path.resolve(__dirname, "../../../node_modules/fflate/lib/node.cjs"),
"@": path.resolve(__dirname, "../../"),
"@lib": path.resolve(__dirname, "../../lib/src"),
fflate: resolve(__dirname, "../../../node_modules/fflate/lib/node.cjs"),
"@": resolve(__dirname, "../../"),
"@lib": resolve(__dirname, "../../lib/src"),
"../../src/worker/bgWorker.ts": "../../src/worker/bgWorker.mock.ts",
},
},
@@ -41,7 +83,7 @@ export default defineConfig({
minify: false,
rollupOptions: {
input: {
index: path.resolve(__dirname, "entrypoint.ts"),
index: resolve(__dirname, "entrypoint.ts"),
},
external: (id) => {
if (defaultExternal.includes(id)) return true;
@@ -57,7 +99,7 @@ export default defineConfig({
},
},
lib: {
entry: path.resolve(__dirname, "entrypoint.ts"),
entry: resolve(__dirname, "entrypoint.ts"),
formats: ["cjs"],
fileName: "index",
},

View File

@@ -41,7 +41,7 @@ async function renderHistoryList(): Promise<VaultHistoryItem[]> {
const [items, lastUsedId] = await Promise.all([historyStore.getVaultHistory(), historyStore.getLastUsedVaultId()]);
listEl.innerHTML = "";
listEl.replaceChildren();
emptyEl.classList.toggle("is-hidden", items.length > 0);
for (const item of items) {

142
src/common/reportTool.ts Normal file
View File

@@ -0,0 +1,142 @@
import { REMOTE_COUCHDB, REMOTE_MINIO } from "@lib/common/models/setting.const";
import type { ObsidianLiveSyncSettings } from "@lib/common/models/setting.type";
import { generateCredentialObject } from "@lib/replication/httplib";
import { parseHeaderValues } from "@lib/common/utils";
import { requestToCouchDBWithCredentials } from "./utils";
import { LOG_LEVEL_VERBOSE, Logger } from "@lib/common/logger";
import { DEFAULT_SETTINGS } from "@lib/common/models/setting.const.defaults";
import { isCloudantURI } from "@lib/pouchdb/utils_couchdb";
import { compatGlobal } from "@lib/common/coreEnvFunctions";
import { manifestVersion, packageVersion } from "@lib/common/coreEnvVars";
import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore";
function redactObject(obj: Record<string, any>, dotted: string, redactedValue = "REDACTED") {
const keys = dotted.split(".");
let current = obj;
for (let i = 0; i < keys.length - 1; i++) {
const key = keys[i];
if (!(key in current)) {
current[key] = {} as Record<string, any>;
}
// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
current = current[key];
}
const lastKey = keys[keys.length - 1];
if (lastKey in current) {
current[lastKey] = redactedValue;
}
return obj;
}
export async function generateReport(settings: ObsidianLiveSyncSettings, core: LiveSyncBaseCore) {
let responseConfig: Record<string, any> = {};
const REDACTED = "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷";
if (settings.remoteType == REMOTE_COUCHDB) {
try {
const credential = generateCredentialObject(settings);
const customHeaders = parseHeaderValues(settings.couchDB_CustomHeaders);
const r = await requestToCouchDBWithCredentials(
settings.couchDB_URI,
credential,
window.origin,
undefined,
undefined,
undefined,
customHeaders
);
responseConfig = r.json as Record<string, any>;
redactObject(responseConfig, "couch_httpd_auth.secret");
redactObject(responseConfig, "couch_httpd_auth.authentication_db");
redactObject(responseConfig, "couch_httpd_auth.authentication_redirect");
redactObject(responseConfig, "couchdb.uuid");
redactObject(responseConfig, "admins");
redactObject(responseConfig, "users");
redactObject(responseConfig, "chttpd_auth.secret");
delete responseConfig["jwt_keys"];
} catch (ex) {
Logger(ex, LOG_LEVEL_VERBOSE);
responseConfig = {
error: "Requesting information from the remote CouchDB has failed. If you are using IBM Cloudant, this is normal behaviour.",
};
}
} else if (settings.remoteType == REMOTE_MINIO) {
responseConfig = { error: "Object Storage Synchronisation" };
//
}
const defaultKeys = Object.keys(DEFAULT_SETTINGS) as (keyof ObsidianLiveSyncSettings)[];
const pluginConfig = JSON.parse(JSON.stringify(settings)) as ObsidianLiveSyncSettings;
const pluginKeys = Object.keys(pluginConfig);
for (const key of pluginKeys) {
if (defaultKeys.includes(key as keyof ObsidianLiveSyncSettings)) continue;
delete pluginConfig[key as keyof ObsidianLiveSyncSettings];
}
pluginConfig.couchDB_DBNAME = REDACTED;
pluginConfig.couchDB_PASSWORD = REDACTED;
const scheme = pluginConfig.couchDB_URI.startsWith("http:")
? "(HTTP)"
: pluginConfig.couchDB_URI.startsWith("https:")
? "(HTTPS)"
: "";
pluginConfig.couchDB_URI = isCloudantURI(pluginConfig.couchDB_URI) ? "cloudant" : `self-hosted${scheme}`;
pluginConfig.couchDB_USER = REDACTED;
pluginConfig.passphrase = REDACTED;
pluginConfig.encryptedPassphrase = REDACTED;
pluginConfig.encryptedCouchDBConnection = REDACTED;
pluginConfig.accessKey = REDACTED;
pluginConfig.secretKey = REDACTED;
const redact = (source: string) => `${REDACTED}(${source.length} letters)`;
const toSchemeOnly = (uri: string) => {
try {
return `${new URL(uri).protocol}//`;
} catch {
const matched = uri.match(/^[A-Za-z][A-Za-z0-9+.-]*:\/\//);
return matched?.[0] ?? REDACTED;
}
};
pluginConfig.remoteConfigurations = Object.fromEntries(
Object.entries(pluginConfig.remoteConfigurations || {}).map(([id, config]) => [
id,
{
...config,
uri: toSchemeOnly(config.uri),
},
])
);
pluginConfig.region = redact(pluginConfig.region);
pluginConfig.bucket = redact(pluginConfig.bucket);
pluginConfig.pluginSyncExtendedSetting = {};
pluginConfig.P2P_AppID = redact(pluginConfig.P2P_AppID);
pluginConfig.P2P_passphrase = redact(pluginConfig.P2P_passphrase);
pluginConfig.P2P_roomID = redact(pluginConfig.P2P_roomID);
pluginConfig.P2P_relays = redact(pluginConfig.P2P_relays);
pluginConfig.jwtKey = redact(pluginConfig.jwtKey);
pluginConfig.jwtSub = redact(pluginConfig.jwtSub);
pluginConfig.jwtKid = redact(pluginConfig.jwtKid);
pluginConfig.bucketCustomHeaders = redact(pluginConfig.bucketCustomHeaders);
pluginConfig.couchDB_CustomHeaders = redact(pluginConfig.couchDB_CustomHeaders);
pluginConfig.P2P_turnCredential = redact(pluginConfig.P2P_turnCredential);
pluginConfig.P2P_turnUsername = redact(pluginConfig.P2P_turnUsername);
pluginConfig.P2P_turnServers = `(${pluginConfig.P2P_turnServers.split(",").length} servers configured)`;
const endpoint = pluginConfig.endpoint;
if (endpoint == "") {
pluginConfig.endpoint = "Not configured or AWS";
} else {
const endpointScheme = pluginConfig.endpoint.startsWith("http:")
? "(HTTP)"
: pluginConfig.endpoint.startsWith("https:")
? "(HTTPS)"
: "";
pluginConfig.endpoint = `${endpoint.indexOf(".r2.cloudflarestorage.") !== -1 ? "R2" : "self-hosted?"}(${endpointScheme})`;
}
const obsidianInfo = {
navigator: compatGlobal.navigator.userAgent,
fileSystem: core.services.vault.isStorageInsensitive() ? "insensitive" : "sensitive",
};
const result = {
obsidianInfo,
responseConfig,
pluginConfig,
manifestVersion,
packageVersion,
};
return result;
}

View File

@@ -128,7 +128,7 @@ export const _requestToCouchDBFetch = async (
username: string,
password: string,
path?: string,
body?: string | any,
body?: any,
method?: string
) => {
const utf8str = String.fromCharCode.apply(null, [...writeString(`${username}:${password}`)]);
@@ -138,7 +138,7 @@ export const _requestToCouchDBFetch = async (
authorization: authHeader,
"content-type": "application/json",
};
const uri = `${baseUri}/${path}`;
const uri = `${baseUri.replace(/\/+$/, "")}/${path}`;
const requestParam = {
url: uri,
method: method || (body ? "PUT" : "GET"),
@@ -146,7 +146,7 @@ export const _requestToCouchDBFetch = async (
contentType: "application/json",
body: JSON.stringify(body),
};
return await fetch(uri, requestParam);
return await _fetch(uri, requestParam);
};
export const _requestToCouchDB = async (
@@ -162,7 +162,7 @@ export const _requestToCouchDB = async (
const authHeaderGen = new AuthorizationHeaderGenerator();
const authHeader = await authHeaderGen.getAuthorizationHeader(credentials);
const transformedHeaders: Record<string, string> = { authorization: authHeader, origin: origin, ...customHeaders };
const uri = `${baseUri}/${path}`;
const uri = `${baseUri.replace(/\/+$/, "")}/${path}`;
const requestParam: RequestUrlParam = {
url: uri,
method: method || (body ? "PUT" : "GET"),
@@ -214,6 +214,7 @@ import { BASE_IS_NEW, EVEN, TARGET_IS_NEW } from "@lib/common/models/shared.cons
export { BASE_IS_NEW, EVEN, TARGET_IS_NEW };
// Why 2000? : ZIP FILE Does not have enough resolution.
import { compareMTime } from "@lib/common/utils.ts";
import { _fetch } from "@/lib/src/common/coreEnvFunctions.ts";
export { compareMTime };
function getKey(file: AnyEntry | string | UXFileInfoStub) {
const key = typeof file == "string" ? file : stripAllPrefixes(file.path);

View File

@@ -68,9 +68,10 @@ import { ConflictResolveModal } from "../../modules/features/InteractiveConflict
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import { EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG, eventHub } from "../../common/events.ts";
import { PluginDialogModal } from "./PluginDialogModal.ts";
import { $msg } from "src/lib/src/common/i18n.ts";
import { $msg } from "@/lib/src/common/i18n.ts";
import type { InjectableServiceHub } from "../../lib/src/services/InjectableServices.ts";
import type { LiveSyncCore } from "../../main.ts";
import { LiveSyncError } from "@lib/common/LSError.ts";
const d = "\u200b";
const d2 = "\n";
@@ -564,7 +565,7 @@ export class ConfigSync extends LiveSyncCommands {
...data,
documentPath: this.getPath(wx),
files: xFiles,
} as PluginDataExDisplay;
} satisfies PluginDataExDisplay;
}
return false;
}
@@ -1069,10 +1070,10 @@ export class ConfigSync extends LiveSyncCommands {
}
const baseDir = this.configDir;
try {
if (!data.documentPath) throw "InternalError: Document path not exist";
if (!data.documentPath) throw new LiveSyncError("InternalError: Document path not exist");
const dx = await this.localDatabase.getDBEntry(data.documentPath);
if (dx == false) {
throw "Not found on database";
throw new LiveSyncError("Not found on database");
}
const loadedData = deserialize(getDocDataAsArray(dx.data), {}) as PluginDataEx;
for (const f of loadedData.files) {
@@ -1317,7 +1318,7 @@ export class ConfigSync extends LiveSyncCommands {
}
const docXDoc = await this.localDatabase.getDBEntryFromMeta(old, false, false);
if (docXDoc == false) {
throw "Could not load the document";
throw new LiveSyncError("Could not load the document");
}
const dataSrc = getDocData(docXDoc.data);
const dataStart = dataSrc.indexOf(DUMMY_END);

View File

@@ -50,6 +50,7 @@ import { hiddenFilesEventCount, hiddenFilesProcessingCount } from "../../lib/src
import { EVENT_SETTING_SAVED, eventHub } from "../../common/events.ts";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import type { LiveSyncCore } from "../../main.ts";
import { tryGetFilePath } from "@lib/common/utils.doc.ts";
type SyncDirection = "push" | "pull" | "safe" | "pullForce" | "pushForce";
declare global {
@@ -317,7 +318,7 @@ export class HiddenFileSync extends LiveSyncCommands {
this._fileInfoLastProcessed.set(file, key);
}
async updateLastProcessedAsActualFile(file: FilePath, stat?: UXStat | null | undefined) {
async updateLastProcessedAsActualFile(file: FilePath, stat?: UXStat | null) {
if (!stat) stat = await this.core.storageAccess.statHidden(file);
this._fileInfoLastProcessed.set(file, this.statToKey(stat));
}
@@ -411,10 +412,7 @@ export class HiddenFileSync extends LiveSyncCommands {
}
}
async updateLastProcessedAsActualDatabase(
file: FilePath,
doc?: MetaEntry | LoadedEntry | null | undefined | false
) {
async updateLastProcessedAsActualDatabase(file: FilePath, doc?: MetaEntry | LoadedEntry | null | false) {
const dbPath = addPrefix(file, ICHeader);
if (!doc) doc = await this.localDatabase.getDBEntryMeta(dbPath);
if (!doc) return;
@@ -1050,7 +1048,7 @@ Offline Changed files: ${processFiles.length}`;
}
notifyProgress();
} catch (ex) {
this._log(`Failed to process storage change file:${file}`, logLevel);
this._log(`Failed to process storage change file:${tryGetFilePath(file)}`, logLevel);
this._log(ex, LOG_LEVEL_VERBOSE);
}
});
@@ -1162,7 +1160,7 @@ Offline Changed files: ${files.length}`;
await this.trackDatabaseFileModification(path, "[Scanning]", true, onlyNew, file);
notifyProgress();
} catch (ex) {
this._log(`Failed to process database changes:${file}`);
this._log(`Failed to process database changes:${tryGetFilePath(file)}`);
this._log(ex, LOG_LEVEL_VERBOSE);
}
return;
@@ -1500,7 +1498,7 @@ Offline Changed files: ${files.length}`;
}
async storeInternalFileToDatabase(file: InternalFileInfo | UXFileInfo, forceWrite = false) {
const storeFilePath = stripAllPrefixes(file.path as FilePath);
const storeFilePath = stripAllPrefixes(file.path);
const storageFilePath = file.path;
if (await this.services.vault.isIgnoredByIgnoreFile(storageFilePath)) {
return undefined;

View File

@@ -16,9 +16,8 @@ import { serialized } from "octagonal-wheels/concurrency/lock_v2";
import { arrayToChunkedArray } from "octagonal-wheels/collection";
import { EVENT_ANALYSE_DB_USAGE, EVENT_REQUEST_PERFORM_GC_V3, eventHub } from "@/common/events";
import type { LiveSyncCouchDBReplicator } from "@/lib/src/replication/couchdb/LiveSyncReplicator";
import { delay, parseHeaderValues } from "@/lib/src/common/utils";
import { generateCredentialObject } from "@/lib/src/replication/httplib";
import { _requestToCouchDB } from "@/common/utils";
import { delay } from "@/lib/src/common/utils";
// import { _requestToCouchDB } from "@/common/utils";
const DB_KEY_SEQ = "gc-seq";
const DB_KEY_CHUNK_SET = "chunk-set";
const DB_KEY_DOC_USAGE_MAP = "doc-usage-map";
@@ -391,7 +390,7 @@ Note: **Make sure to synchronise all devices before deletion.**
.map((revInfo) => db.get(doc._id, { rev: revInfo.rev }))
).then((docs) => docs.filter((doc) => doc));
for (const oldDoc of oldDocs) {
await processDoc(oldDoc as EntryDoc, false);
await processDoc(oldDoc, false);
}
}
} catch (ex) {
@@ -533,7 +532,7 @@ Success: ${successCount}, Errored: ${errored}`;
const docMap = new Map<DocumentID, Set<DocumentInfo>>();
const info = await db.info();
// Total number of revisions to process (approximate)
const maxSeq = new Number(info.update_seq);
const maxSeq = Number.parseInt(`${info.update_seq ?? 0}`, 10);
let processed = 0;
let read = 0;
let errored = 0;
@@ -560,7 +559,7 @@ Success: ${successCount}, Errored: ${errored}`;
});
docMap.set(id, set);
} else if (doc.type === EntryTypes.CHUNK) {
const id = doc._id as DocumentID;
const id = doc._id;
if (chunkMap.has(id)) {
return;
}
@@ -759,67 +758,68 @@ Success: ${successCount}, Errored: ${errored}`;
}
}
/**
* Compact the database by temporarily setting the revision limit to 1.
* @returns
*/
async compactDatabaseWithRevLimit() {
// Temporarily set revs_limit to 1, perform compaction, and restore the original revs_limit.
// Very dangerous operation, so now suppressed.
return false;
const replicator = this.core.replicator as LiveSyncCouchDBReplicator;
const remote = await replicator.connectRemoteCouchDBWithSetting(this.settings, false, false, true);
if (!remote) {
this._notice("Failed to connect to remote for compaction.");
return;
}
if (typeof remote == "string") {
this._notice(`Failed to connect to remote for compaction. ${remote}`);
return;
}
const customHeaders = parseHeaderValues(this.settings.couchDB_CustomHeaders);
const credential = generateCredentialObject(this.settings);
const request = async (path: string, method: string = "GET", body: any = undefined) => {
const req = await _requestToCouchDB(
this.settings.couchDB_URI + (this.settings.couchDB_DBNAME ? `/${this.settings.couchDB_DBNAME}` : ""),
credential,
window.origin,
path,
body,
method,
customHeaders
);
return req;
};
let revsLimit = "";
const req = await request(`_revs_limit`, "GET");
if (req.status == 200) {
revsLimit = req.text.trim();
this._info(`Remote database _revs_limit: ${revsLimit}`);
} else {
this._notice(`Failed to get remote database _revs_limit. Status: ${req.status}`);
return;
}
const req2 = await request(`_revs_limit`, "PUT", 1);
if (req2.status == 200) {
this._info(`Set remote database _revs_limit to 1 for compaction.`);
}
try {
await this.compactDatabase();
} finally {
// Restore revs_limit
if (revsLimit) {
const req3 = await request(`_revs_limit`, "PUT", parseInt(revsLimit));
if (req3.status == 200) {
this._info(`Restored remote database _revs_limit to ${revsLimit}.`);
} else {
this._notice(
`Failed to restore remote database _revs_limit. Status: ${req3.status} / ${req3.text}`
);
}
}
}
}
// /**
// * Compact the database by temporarily setting the revision limit to 1.
// * @returns
// */
// async compactDatabaseWithRevLimit() {
// // Temporarily set revs_limit to 1, perform compaction, and restore the original revs_limit.
// // Very dangerous operation, so now suppressed.
// return Promise.resolve(false);
// const replicator = this.core.replicator as LiveSyncCouchDBReplicator;
// const remote = await replicator.connectRemoteCouchDBWithSetting(this.settings, false, false, true);
// if (!remote) {
// this._notice("Failed to connect to remote for compaction.");
// return;
// }
// if (typeof remote == "string") {
// this._notice(`Failed to connect to remote for compaction. ${remote}`);
// return;
// }
// const customHeaders = parseHeaderValues(this.settings.couchDB_CustomHeaders);
// const credential = generateCredentialObject(this.settings);
// const request = async (path: string, method: string = "GET", body: any = undefined) => {
// const req = await _requestToCouchDB(
// this.settings.couchDB_URI.replace(/\/+$/, "") +
// (this.settings.couchDB_DBNAME ? `/${this.settings.couchDB_DBNAME}` : ""),
// credential,
// window.origin,
// path,
// body,
// method,
// customHeaders
// );
// return req;
// };
// let revsLimit = "";
// const req = await request(`_revs_limit`, "GET");
// if (req.status == 200) {
// revsLimit = req.text.trim();
// this._info(`Remote database _revs_limit: ${revsLimit}`);
// } else {
// this._notice(`Failed to get remote database _revs_limit. Status: ${req.status}`);
// return;
// }
// const req2 = await request(`_revs_limit`, "PUT", 1);
// if (req2.status == 200) {
// this._info(`Set remote database _revs_limit to 1 for compaction.`);
// }
// try {
// await this.compactDatabase();
// } finally {
// // Restore revs_limit
// if (revsLimit) {
// const req3 = await request(`_revs_limit`, "PUT", parseInt(revsLimit));
// if (req3.status == 200) {
// this._info(`Restored remote database _revs_limit to ${revsLimit}.`);
// } else {
// this._notice(
// `Failed to restore remote database _revs_limit. Status: ${req3.status} / ${req3.text}`
// );
// }
// }
// }
// }
async gcv3() {
if (!this.isAvailable()) return;
const replicator = this.core.replicator as LiveSyncCouchDBReplicator;
@@ -928,7 +928,7 @@ This may indicate that some devices have not completed synchronisation, which co
usedChunks.add(chunkId);
}
} else if (doc.type === EntryTypes.CHUNK) {
allChunks.set(doc._id as DocumentID, doc._rev);
allChunks.set(doc._id, doc._rev);
}
}
this._notice(

View File

@@ -0,0 +1,80 @@
import { App, Modal } from "@/deps.ts";
import P2POpenReplicationPane from "./P2POpenReplicationPane.svelte";
import { mount, unmount } from "svelte";
import type { LiveSyncTrysteroReplicator } from "@/lib/src/replication/trystero/LiveSyncTrysteroReplicator";
export type P2POpenReplicationModalCallback = {
onSync: (peerId: string) => Promise<void>;
onSyncAndClose: (peerId: string) => Promise<void>;
};
export class P2POpenReplicationModal extends Modal {
liveSyncReplicator: LiveSyncTrysteroReplicator;
callback?: P2POpenReplicationModalCallback;
component?: ReturnType<typeof mount>;
showResult: boolean;
title: string;
onClosed?: () => void;
rebuildMode: boolean;
constructor(
app: App,
liveSyncReplicator: LiveSyncTrysteroReplicator,
callback?: P2POpenReplicationModalCallback,
showResult: boolean = false,
title: string = "P2P Replication",
onClosed?: () => void,
rebuildMode: boolean = false
) {
super(app);
this.liveSyncReplicator = liveSyncReplicator;
this.callback = callback;
this.showResult = showResult;
this.title = title;
this.onClosed = onClosed;
this.rebuildMode = rebuildMode;
}
async onSync(peerId: string) {
if (this.callback?.onSync) {
await this.callback.onSync(peerId);
}
}
async onSyncAndClose(peerId: string) {
if (this.callback?.onSyncAndClose) {
await this.callback.onSyncAndClose(peerId);
}
this.close();
}
override onOpen() {
const { contentEl } = this;
this.titleEl.setText(this.title);
contentEl.empty();
if (this.component === undefined) {
this.component = mount(P2POpenReplicationPane, {
target: contentEl,
props: {
liveSyncReplicator: this.liveSyncReplicator,
onSync: (peerId: string) => this.onSync(peerId),
onSyncAndClose: (peerId: string) => this.onSyncAndClose(peerId),
onClose: () => this.close(),
showResult: this.showResult,
rebuildMode: this.rebuildMode,
},
});
}
}
override onClose() {
const { contentEl } = this;
contentEl.empty();
if (this.component !== undefined) {
void unmount(this.component);
this.component = undefined;
}
this.onClosed?.();
}
}

View File

@@ -0,0 +1,313 @@
<script lang="ts">
import { onMount } from "svelte";
import { eventHub } from "@/common/events";
import {
EVENT_SERVER_STATUS,
EVENT_REQUEST_STATUS,
type P2PServerInfo,
} from "@lib/replication/trystero/TrysteroReplicatorP2PServer";
// import type { TrysteroReplicator } from "@lib/replication/trystero/TrysteroReplicator";
import { LOG_LEVEL_NOTICE, LOG_LEVEL_INFO } from "@lib/common/types";
import { Logger } from "@lib/common/logger";
import type { LiveSyncTrysteroReplicator } from "@/lib/src/replication/trystero/LiveSyncTrysteroReplicator";
import { delay, fireAndForget } from "octagonal-wheels/promises";
import P2PServerStatusCard from "./P2PServerStatusCard.svelte";
interface Props {
liveSyncReplicator: LiveSyncTrysteroReplicator;
onSync: (_peerId: string) => Promise<void>;
onSyncAndClose: (_peerId: string) => Promise<void>;
onClose: () => void;
showResult: boolean;
rebuildMode?: boolean;
}
let { onSync, onSyncAndClose, onClose, showResult, liveSyncReplicator, rebuildMode = false }: Props = $props();
let serverInfo = $state<P2PServerInfo | undefined>(undefined);
let syncingPeerId = $state<string | null>(null);
const logLevel = showResult ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
async function requestServerStatus() {
await liveSyncReplicator.requestStatus();
eventHub.emitEvent(EVENT_REQUEST_STATUS);
}
onMount(() => {
// ServerStatus
const unsubscribe = eventHub.onEvent(EVENT_SERVER_STATUS, (status) => {
serverInfo = status;
});
fireAndForget(async () => {
await delay(100);
await requestServerStatus();
});
return unsubscribe;
});
async function handleSync(peerId: string) {
try {
syncingPeerId = peerId;
Logger(`Starting sync with ${peerId}`, logLevel);
await onSync(peerId);
Logger(`Sync completed with ${peerId}`, logLevel);
} catch (e) {
Logger(`Error during sync: ${e instanceof Error ? e.message : String(e)}`, logLevel);
} finally {
syncingPeerId = null;
}
}
async function handleSyncThenClose(peerId: string) {
try {
syncingPeerId = peerId;
Logger(`Starting sync with ${peerId}`, logLevel);
await onSyncAndClose(peerId);
Logger(`Sync completed with ${peerId}`, logLevel);
} catch (e) {
Logger(`Error during sync: ${e instanceof Error ? e.message : String(e)}`, logLevel);
} finally {
syncingPeerId = null;
}
}
async function handleSyncAndClose(peerId: string) {
fireAndForget(async () => {
try {
Logger(`Starting sync with ${peerId}`, logLevel);
await onSync(peerId);
Logger(`Sync completed with ${peerId}`, logLevel);
} catch (e) {
Logger(`Error during sync: ${e instanceof Error ? e.message : String(e)}`, logLevel);
}
});
onClose();
}
async function disconnect() {
try {
await liveSyncReplicator.close();
Logger("Signalling connection closed.", logLevel);
} catch (e) {
Logger(`Failed to close signalling connection: ${e instanceof Error ? e.message : String(e)}`, logLevel);
}
}
async function onCloseAndDisconnect() {
await disconnect();
onClose();
}
function getAcceptanceStatus(peer: P2PServerInfo["knownAdvertisements"][number]) {
if (peer.isTemporaryAccepted === true) return "ACCEPTED (in session)";
if (peer.isAccepted === true) return "ACCEPTED";
if (peer.isTemporaryAccepted === false) return "DENIED (in session)";
if (peer.isAccepted === false) return "DENIED";
return "NEW";
}
function getAcceptanceStatusClass(peer: P2PServerInfo["knownAdvertisements"][number]) {
if (peer.isTemporaryAccepted === true || peer.isAccepted === true) return "accepted";
if (peer.isTemporaryAccepted === false || peer.isAccepted === false) return "denied";
return "unknown";
}
</script>
<div class="p2p-container">
<P2PServerStatusCard {liveSyncReplicator} showBroadcastToggle={false} />
<div class="peers-section">
<h3>Available Peers</h3>
{#if serverInfo && serverInfo.knownAdvertisements.length > 0}
<div class="peers-list">
{#each serverInfo.knownAdvertisements as peer (peer.peerId)}
<div class="peer-item">
<div class="peer-info">
<div class="peer-name">{peer.name}</div>
<div class="peer-meta">
<span class="badge">{peer.platform}</span>
<span class="peer-id-mini" title={peer.peerId}>
{peer.peerId.slice(0, 8)}
</span>
<span class="badge status-chip {getAcceptanceStatusClass(peer)}">
{getAcceptanceStatus(peer)}
</span>
</div>
</div>
<div class="peer-actions">
{#if !rebuildMode}
<button
class="btn btn-primary"
disabled={syncingPeerId !== null}
onclick={() => handleSync(peer.peerId)}
>
{syncingPeerId === peer.peerId ? "Syncing..." : "Sync"}
</button>
<button
class="btn {rebuildMode ? 'btn-primary' : 'btn-secondary'}"
disabled={syncingPeerId !== null}
onclick={() => handleSyncAndClose(peer.peerId)}
>
{syncingPeerId === peer.peerId ? "Syncing..." : "Start Sync & Close"}
</button>
{:else}
<button
class="btn {rebuildMode ? 'btn-primary' : 'btn-secondary'}"
disabled={syncingPeerId !== null}
onclick={() => handleSyncThenClose(peer.peerId)}
>
{syncingPeerId === peer.peerId ? "Syncing..." : "Sync"}
</button>
{/if}
</div>
</div>
{/each}
</div>
{:else if serverInfo}
<p class="no-peers">No devices available. Waiting for other devices to connect...</p>
{/if}
</div>
<div class="footer">
{#if rebuildMode}
<button class="btn btn-cancel" onclick={onClose} disabled={syncingPeerId !== null}>Skip and close</button>
{:else}
<button class="btn btn-cancel" onclick={onClose}>Close</button>
<button class="btn btn-cancel" onclick={onCloseAndDisconnect}>Close & Disconnect</button>
{/if}
</div>
</div>
<style>
.p2p-container {
display: flex;
flex-direction: column;
gap: 1rem;
padding: 1rem;
max-height: 70vh;
overflow-y: auto;
}
.peers-section {
border: 1px solid var(--divider-color);
border-radius: 0.5rem;
padding: 1rem;
}
h3 {
margin: 0 0 0.75rem 0;
font-weight: 600;
font-size: 1rem;
}
.peers-list {
display: flex;
flex-direction: column;
gap: 0.5rem;
}
.peer-item {
display: flex;
justify-content: space-between;
flex-wrap: wrap;
align-items: center;
padding: 0.75rem;
background-color: var(--background-secondary);
border: 1px solid var(--divider-color);
border-radius: 0.4rem;
}
.peer-info {
flex: 1;
}
.peer-name {
font-weight: 600;
margin-bottom: 0.25rem;
}
.peer-meta {
display: flex;
gap: 0.5rem;
font-size: 0.8rem;
}
.badge {
background-color: var(--background-tertiary);
padding: 0.1rem 0.4rem;
border-radius: 0.25rem;
}
.status-chip {
font-weight: 600;
}
.status-chip.accepted {
background-color: var(--background-modifier-success);
color: var(--text-normal);
}
.status-chip.denied {
background-color: var(--background-modifier-error);
color: var(--text-normal);
}
.status-chip.unknown {
background-color: var(--background-modifier-border);
color: var(--text-muted);
}
.peer-id-mini {
font-family: monospace;
color: var(--text-muted);
}
.peer-actions {
flex-wrap: wrap;
display: flex;
gap: 0.5rem;
}
.btn {
padding: 0.4rem 0.8rem;
border: 1px solid var(--divider-color);
border-radius: 0.3rem;
background-color: var(--interactive-normal);
color: var(--text-normal);
cursor: pointer;
font-size: 0.8rem;
font-weight: 500;
transition: background-color 0.2s;
}
.btn:hover:not(:disabled) {
background-color: var(--interactive-hover);
}
.btn:disabled {
opacity: 0.6;
cursor: not-allowed;
}
.btn-primary {
background-color: var(--interactive-accent);
color: var(--text-on-accent);
}
.btn-secondary {
background-color: var(--background-tertiary);
}
.btn-cancel {
width: 100%;
margin-top: 0.5rem;
}
.no-peers {
text-align: center;
color: var(--text-muted);
font-size: 0.9rem;
padding: 1rem;
}
.footer {
border-top: 1px solid var(--divider-color);
padding-top: 0.75rem;
}
</style>

View File

@@ -0,0 +1,131 @@
import { App } from "@/deps.ts";
import { Logger } from "@lib/common/logger";
import { LOG_LEVEL_NOTICE, LOG_LEVEL_INFO } from "@lib/common/types";
import type { LiveSyncTrysteroReplicator } from "@lib/replication/trystero/LiveSyncTrysteroReplicator";
import { P2POpenReplicationModal } from "./P2POpenReplicationModal";
/**
* Creates an openReplicationUI factory for Obsidian environments.
* Returns a per-replicator closure that opens the P2P Replication modal
* and performs bidirectional sync (pull then push on success).
*
* Usage:
* const factory = createOpenReplicationUI(app);
* useP2PReplicatorFeature(core, factory);
*/
export function createOpenReplicationUI(
app: App
): (replicator: LiveSyncTrysteroReplicator) => (showResult: boolean) => Promise<boolean | void> {
return (replicator: LiveSyncTrysteroReplicator) =>
(showResult: boolean): Promise<boolean | void> => {
const logLevel = showResult ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
return new Promise<boolean | void>((resolve) => {
const modal = new P2POpenReplicationModal(
app,
replicator,
{
onSync: async (peerId: string) => {
try {
// pull (replicateFrom) first; push only on success
const pullResult = await replicator.replicateFrom(peerId, showResult);
if (pullResult?.ok) {
const pushResult = await replicator.requestSynchroniseToPeer(peerId);
resolve(pushResult?.ok ?? true);
} else {
resolve(false);
}
} catch (e) {
Logger(
`Error in bidirectional sync with ${peerId}: ${e instanceof Error ? e.message : String(e)}`,
logLevel
);
resolve(false);
}
},
onSyncAndClose: async (peerId: string) => {
try {
const pullResult = await replicator.replicateFrom(peerId, showResult);
if (pullResult?.ok) {
const pushResult = await replicator.requestSynchroniseToPeer(peerId);
if (pushResult?.ok ?? true) {
await replicator.close();
resolve(true);
} else {
resolve(false);
}
} else {
resolve(false);
}
} catch (e) {
Logger(
`Error in bidirectional sync with ${peerId}: ${e instanceof Error ? e.message : String(e)}`,
logLevel
);
resolve(false);
}
},
},
showResult
);
modal.open();
});
};
}
/**
* Creates an openRebuildUI factory for Obsidian environments.
* Opens the P2P Replication modal in "rebuild" mode — one-way pull only,
* with setOnSetup / clearOnSetup bracketing the replicateFrom call.
*
* Usage:
* const factory = createOpenRebuildUI(app);
* useP2PReplicatorFeature(core, createOpenReplicationUI(app), factory);
*/
export function createOpenRebuildUI(
app: App
): (replicator: LiveSyncTrysteroReplicator) => (showResult: boolean) => Promise<boolean | void> {
return (replicator: LiveSyncTrysteroReplicator) =>
(showResult: boolean): Promise<boolean | void> => {
const logLevel = showResult ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
return new Promise<boolean | void>((resolve) => {
let resolved = false;
const safeResolve = (val: boolean) => {
if (!resolved) {
resolved = true;
resolve(val);
}
};
const doRebuild = async (peerId: string) => {
replicator.setOnSetup();
try {
Logger(`Rebuilding from peer ${peerId}`, logLevel);
const result = await replicator.replicateFrom(peerId, showResult);
safeResolve(result?.ok ?? false);
} catch (e) {
Logger(
`Error in rebuild from ${peerId}: ${e instanceof Error ? e.message : String(e)}`,
logLevel
);
safeResolve(false);
} finally {
replicator.clearOnSetup();
}
};
const modal = new P2POpenReplicationModal(
app,
replicator,
{
onSync: doRebuild,
onSyncAndClose: doRebuild,
},
showResult,
"P2P Rebuild",
() => safeResolve(false),
true
);
modal.open();
});
};
}

View File

@@ -5,20 +5,21 @@
AcceptedStatus,
ConnectionStatus,
type PeerStatus,
} from "../../../lib/src/replication/trystero/P2PReplicatorPaneCommon";
import type { LiveSyncTrysteroReplicator } from "../../../lib/src/replication/trystero/LiveSyncTrysteroReplicator";
} from "@lib/replication/trystero/P2PReplicatorPaneCommon";
import type { LiveSyncTrysteroReplicator } from "@lib/replication/trystero/LiveSyncTrysteroReplicator";
import PeerStatusRow from "../P2PReplicator/PeerStatusRow.svelte";
import { EVENT_LAYOUT_READY, eventHub } from "../../../common/events";
import { EVENT_LAYOUT_READY, eventHub } from "@/common/events";
import {
type PeerInfo,
type P2PServerInfo,
EVENT_SERVER_STATUS,
EVENT_REQUEST_STATUS,
EVENT_P2P_REPLICATOR_STATUS,
} from "../../../lib/src/replication/trystero/TrysteroReplicatorP2PServer";
import { type P2PReplicatorStatus } from "../../../lib/src/replication/trystero/TrysteroReplicator";
import { $msg as _msg } from "../../../lib/src/common/i18n";
import { SETTING_KEY_P2P_DEVICE_NAME } from "../../../lib/src/common/types";
} from "@lib/replication/trystero/TrysteroReplicatorP2PServer";
import { type P2PReplicatorStatus } from "@lib/replication/trystero/TrysteroReplicator";
import { $msg as _msg } from "@lib/common/i18n";
import { SETTING_KEY_P2P_DEVICE_NAME } from "@lib/common/types";
import { generateP2PRoomId } from "@lib/common/utils";
import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore";
interface Props {
@@ -148,6 +149,7 @@
eventHub.emitEvent(EVENT_REQUEST_STATUS);
return () => {
r();
rx();
r2();
r3();
};
@@ -216,18 +218,8 @@
function useDefaultRelay() {
eRelay = DEFAULT_SETTINGS.P2P_relays;
}
function _generateRandom() {
return (Math.floor(Math.random() * 1000) + 1000).toString().substring(1);
}
function generateRandom(length: number) {
let buf = "";
while (buf.length < length) {
buf += "-" + _generateRandom();
}
return buf.substring(1, length);
}
function chooseRandom() {
eRoomId = generateRandom(12) + "-" + Math.random().toString(36).substring(2, 5);
eRoomId = generateP2PRoomId();
}
async function openServer() {
@@ -251,7 +243,7 @@
setting?: boolean;
};
return initialDialogStatus;
} catch (e) {
} catch {
return {};
}
};

View File

@@ -0,0 +1,310 @@
<script lang="ts">
import { onMount } from "svelte";
import { eventHub } from "@/common/events";
import { delay, fireAndForget } from "octagonal-wheels/promises";
import type { P2PServerInfo } from "@lib/replication/trystero/TrysteroReplicatorP2PServer";
import {
EVENT_SERVER_STATUS,
EVENT_REQUEST_STATUS,
EVENT_P2P_REPLICATOR_STATUS,
} from "@lib/replication/trystero/TrysteroReplicatorP2PServer";
import { EVENT_SETTING_SAVED } from "@lib/events/coreEvents";
import type { LiveSyncTrysteroReplicator } from "@/lib/src/replication/trystero/LiveSyncTrysteroReplicator";
import type { P2PReplicatorStatus } from "@/lib/src/replication/trystero/TrysteroReplicator";
import { extractP2PRoomSuffix } from "@/lib/src/common/utils";
import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore";
interface Props {
liveSyncReplicator: LiveSyncTrysteroReplicator;
showBroadcastToggle?: boolean;
core?: LiveSyncBaseCore;
}
let { liveSyncReplicator, showBroadcastToggle = true, core }: Props = $props();
let serverInfo = $state<P2PServerInfo | undefined>(undefined);
let replicatorStatus = $state<P2PReplicatorStatus | undefined>(undefined);
let roomSuffix = $state<string>(extractP2PRoomSuffix(core?.services.setting.currentSettings()?.P2P_roomID ?? ""));
let useDiagRTC = $state<boolean>(core?.services.setting.currentSettings()?.P2P_useDiagRTC ?? false);
async function requestServerStatus() {
await Promise.resolve(liveSyncReplicator.requestStatus());
eventHub.emitEvent(EVENT_REQUEST_STATUS);
}
async function onOpenConnection() {
await liveSyncReplicator.makeSureOpened();
await requestServerStatus();
}
async function onDisconnect() {
await liveSyncReplicator.close();
await requestServerStatus();
}
function toggleBroadcast() {
if (replicatorStatus?.isBroadcasting) {
liveSyncReplicator.disableBroadcastChanges();
} else {
liveSyncReplicator.enableBroadcastChanges();
}
}
async function toggleDiagRTC() {
if (!core) {
return;
}
const next = !useDiagRTC;
await core.services.setting.updateSettings((settings) => {
settings.P2P_useDiagRTC = next;
return settings;
}, true);
useDiagRTC = next;
}
onMount(() => {
const unsubscribe = eventHub.onEvent(EVENT_SERVER_STATUS, (status) => {
serverInfo = status;
roomSuffix = extractP2PRoomSuffix(status?.roomId ?? "");
});
const unsubscribeStatus = eventHub.onEvent(EVENT_P2P_REPLICATOR_STATUS, (status) => {
replicatorStatus = status;
});
const unsubscribeSettings = eventHub.onEvent(EVENT_SETTING_SAVED, (settings) => {
roomSuffix = extractP2PRoomSuffix(settings?.P2P_roomID ?? "");
useDiagRTC = settings?.P2P_useDiagRTC ?? false;
});
fireAndForget(async () => {
await delay(100);
await requestServerStatus();
});
return () => {
unsubscribe();
unsubscribeStatus();
unsubscribeSettings();
};
});
const isConnected = $derived.by(() => serverInfo?.isConnected);
const isBroadcasting = $derived.by(() => replicatorStatus?.isBroadcasting ?? false);
</script>
<div class="server-status">
<h3>Signalling Status</h3>
<div class="status-item">
<span>Connection:</span>
<span class="status-value {isConnected ? 'connected' : 'disconnected'}">
{isConnected ? "🟢 Connected" : "🔴 Disconnected"}
</span>
</div>
<div class="status-item status-action">
{#if !isConnected}
<button onclick={onOpenConnection}>Open connection</button>
{:else}
<button onclick={onDisconnect}>Close connection</button>
{/if}
</div>
{#if serverInfo}
<div class="status-item">
<span>Room ID suffix:</span>
<span class="room-suffix-display" title={roomSuffix || "Not configured"}>
{roomSuffix || "-"}
</span>
</div>
<div class="status-item">
<span>Peer ID:</span>
<span class="peer-id-display" title={serverInfo.serverPeerId}>
{serverInfo.serverPeerId.slice(0, 12)}...
</span>
</div>
<div class="status-item">
<span>Devices:</span>
<span>{serverInfo.knownAdvertisements.length}</span>
</div>
{/if}
{#if showBroadcastToggle}
<div class="status-item status-action broadcast-row">
<!-- Live-push to peers: stream this device's changes to connected peers for LiveSync -->
<label class="broadcast-label" for="broadcast-toggle">
Live-push to peers
</label>
<button
id="broadcast-toggle"
class="broadcast-button {isBroadcasting ? 'is-on' : 'is-off'}"
onclick={toggleBroadcast}
title={isBroadcasting ? 'Pushing changes to peers — click to stop' : 'Start pushing changes to peers'}
>
{isBroadcasting ? '📡 On' : '📡 Off'}
</button>
</div>
{/if}
{#if core}
<div class="status-item status-action diag-toggle-row">
<label class="broadcast-label" for="diag-toggle">
🕵️ Diag
</label>
<button
id="diag-toggle"
class="broadcast-button {useDiagRTC ? 'is-on' : 'is-off'}"
onclick={toggleDiagRTC}
title={useDiagRTC
? 'Diagnostic RTCPeerConnection is enabled'
: 'Use Diagnostic RTCPeerConnection for statistics'}
>
{useDiagRTC ? 'On' : 'Off'}
</button>
</div>
{/if}
{#if serverInfo}
<div class="diag-section">
<h4>Stats</h4>
<div class="diag-grid">
<div class="diag-item">
<span>Incoming:</span>
<span>{serverInfo.diag.totalNewConnections}</span>
</div>
<div class="diag-item">
<span>Connected:</span>
<span>{serverInfo.diag.totalSuccessfulConnections}</span>
</div>
<div class="diag-item">
<span>Failed:</span>
<span>{serverInfo.diag.totalFailedConnections}</span>
</div>
<div class="diag-item">
<span>Closed:</span>
<span>{serverInfo.diag.totalClosedConnections}</span>
</div>
</div>
</div>
{/if}
</div>
<style>
.server-status {
border: 1px solid var(--divider-color);
border-radius: 0.5rem;
padding: 1rem;
}
h3 {
margin: 0 0 0.75rem 0;
font-weight: 600;
font-size: 1rem;
}
.status-item {
display: flex;
justify-content: space-between;
flex-wrap: wrap;
font-size: 0.9rem;
margin-bottom: 0.5rem;
}
.status-action {
align-items: center;
gap: 0.5rem;
}
.status-value {
font-weight: 500;
}
.status-value.connected {
color: var(--text-success);
}
.status-value.disconnected {
color: var(--text-error);
}
.peer-id-display {
font-family: monospace;
font-size: 0.85rem;
max-width: 150px;
overflow: hidden;
text-overflow: ellipsis;
}
.room-suffix-display {
font-family: monospace;
font-size: 0.85rem;
font-weight: 600;
}
.broadcast-row {
align-items: center;
margin-top: 0.25rem;
}
.diag-toggle-row {
align-items: center;
margin-top: 0.25rem;
}
.broadcast-label {
font-size: 0.9rem;
color: var(--text-normal);
cursor: pointer;
}
.broadcast-button {
font-size: 0.8rem;
padding: 0.2rem 0.6rem;
border: 1px solid var(--divider-color);
border-radius: 0.4rem;
cursor: pointer;
font-weight: 600;
transition: background-color 0.15s;
}
.broadcast-button.is-on {
background-color: var(--interactive-accent);
color: var(--text-on-accent);
border-color: var(--interactive-accent);
}
.broadcast-button.is-off {
background-color: var(--interactive-normal);
color: var(--text-muted);
}
.broadcast-button.is-off:hover {
background-color: var(--interactive-hover);
color: var(--text-normal);
}
.diag-section {
border-top: 1px solid var(--divider-color);
margin-top: 0.75rem;
padding-top: 0.75rem;
}
.diag-section h4 {
margin: 0 0 0.5rem 0;
font-size: 0.9rem;
font-weight: 600;
}
.diag-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(130px, 1fr));
gap: 0.35rem 0.75rem;
}
.diag-item {
display: flex;
justify-content: space-between;
font-size: 0.85rem;
gap: 0.5rem;
}
</style>

View File

@@ -0,0 +1,891 @@
<script lang="ts">
import { onMount } from "svelte";
import { EVENT_LAYOUT_READY, EVENT_REQUEST_OPEN_P2P_SETTINGS, eventHub } from "@/common/events";
import {
EVENT_SERVER_STATUS,
EVENT_REQUEST_STATUS,
EVENT_P2P_REPLICATOR_STATUS,
EVENT_P2P_REPLICATOR_PROGRESS,
type P2PServerInfo,
} from "@lib/replication/trystero/TrysteroReplicatorP2PServer";
import type { LiveSyncTrysteroReplicator } from "@lib/replication/trystero/LiveSyncTrysteroReplicator";
import type { P2PReplicatorStatus, P2PReplicationReport } from "@lib/replication/trystero/TrysteroReplicator";
import { delay, fireAndForget } from "octagonal-wheels/promises";
import P2PServerStatusCard from "./P2PServerStatusCard.svelte";
import { EVENT_SETTING_SAVED } from "@lib/events/coreEvents";
import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore";
import { ConnectionStringParser } from "@lib/common/ConnectionString";
import type { P2PSyncSetting, RemoteConfiguration } from "@lib/common/models/setting.type";
import { activateP2PRemoteConfiguration, createRemoteConfigurationId } from "@lib/serviceFeatures/remoteConfig";
import { extractP2PRoomSuffix } from "@lib/common/utils";
import { SetupManager } from "@/modules/features/SetupManager";
import SetupRemoteP2P from "@/modules/features/SetupWizard/dialogs/SetupRemoteP2P.svelte";
interface Props {
liveSyncReplicator: LiveSyncTrysteroReplicator;
core: LiveSyncBaseCore;
}
let { liveSyncReplicator, core }: Props = $props();
let serverInfo = $state<P2PServerInfo | undefined>(undefined);
let replicatorInfo = $state<P2PReplicatorStatus | undefined>(undefined);
let decidingPeerId = $state<string | null>(null);
let replicatingPeerId = $state<string | null>(null);
let communicatingUntil = $state<Record<string, number>>({});
const COMMUNICATION_HOLD_MS = 2500;
let syncOnReplicationSetting = $state(core.services.setting.currentSettings()?.P2P_SyncOnReplication ?? "");
type P2PRemoteOption = {
id: string;
name: string;
roomSuffix: string;
};
let p2pRemoteOptions = $state<P2PRemoteOption[]>([]);
let selectedP2PRemoteConfigurationId = $state(
core.services.setting.currentSettings()?.P2P_ActiveRemoteConfigurationId ?? ""
);
let selectingP2PRemote = $state(false);
function addToList(item: string, list: string): string {
const items = list
.split(",")
.map((e) => e.trim())
.filter((e) => e);
if (!items.includes(item)) items.push(item);
return items.join(",");
}
function removeFromList(item: string, list: string): string {
return list
.split(",")
.map((e) => e.trim())
.filter((e) => e && e !== item)
.join(",");
}
function markCommunicating(peerId: string) {
const expiry = Date.now() + COMMUNICATION_HOLD_MS;
communicatingUntil = { ...communicatingUntil, [peerId]: expiry };
window.setTimeout(() => {
if ((communicatingUntil[peerId] ?? 0) <= Date.now()) {
const { [peerId]: _removed, ...rest } = communicatingUntil;
communicatingUntil = rest;
}
}, COMMUNICATION_HOLD_MS + 100);
}
function listP2PRemoteOptions(
remoteConfigurations: Record<string, RemoteConfiguration> | undefined
): P2PRemoteOption[] {
return Object.values(remoteConfigurations ?? {})
.map((config) => {
try {
const parsed = ConnectionStringParser.parse(config.uri);
if (parsed.type !== "p2p") {
return undefined;
}
return {
id: config.id,
name: config.name,
roomSuffix: extractP2PRoomSuffix(parsed.settings.P2P_roomID ?? ""),
} as P2PRemoteOption;
} catch {
return undefined;
}
})
.filter((e): e is P2PRemoteOption => !!e);
}
function refreshP2PRemoteOptions() {
const settings = core.services.setting.currentSettings();
const options = listP2PRemoteOptions(settings.remoteConfigurations);
p2pRemoteOptions = options;
const currentSelected = settings.P2P_ActiveRemoteConfigurationId ?? "";
const isCurrentSelectedValid = options.some((option) => option.id === currentSelected);
if (options.length === 0) {
selectedP2PRemoteConfigurationId = "";
return;
}
if (currentSelected.trim() === "" || !isCurrentSelectedValid) {
const fallbackId = options[0].id;
selectedP2PRemoteConfigurationId = fallbackId;
if (currentSelected !== fallbackId) {
fireAndForget(() => applyP2PActiveRemoteSelection(fallbackId));
}
return;
}
selectedP2PRemoteConfigurationId = currentSelected;
}
function canEditP2PSettings() {
const selected = selectedP2PRemoteConfigurationId.trim();
if (selected === "") {
return false;
}
return p2pRemoteOptions.some((e) => e.id === selected);
}
async function requestServerStatus() {
await liveSyncReplicator.requestStatus();
eventHub.emitEvent(EVENT_REQUEST_STATUS);
}
onMount(() => {
const unsubscribe = eventHub.onEvent(EVENT_SERVER_STATUS, (status) => {
serverInfo = status;
});
const unsubscribeReplicatorStatus = eventHub.onEvent(EVENT_P2P_REPLICATOR_STATUS, (status) => {
replicatorInfo = status;
for (const peerId of status.replicatingFrom) {
markCommunicating(peerId);
}
for (const peerId of status.replicatingTo) {
markCommunicating(peerId);
}
});
const unsubscribeReplicatorProgress = eventHub.onEvent(EVENT_P2P_REPLICATOR_PROGRESS, (report) => {
const rep = report as P2PReplicationReport;
if (("fetching" in rep && rep.fetching?.isActive) || ("sending" in rep && rep.sending?.isActive)) {
markCommunicating(rep.peerId);
}
});
const unsubscribeSettings = eventHub.onEvent(EVENT_SETTING_SAVED, (settings) => {
syncOnReplicationSetting = settings?.P2P_SyncOnReplication ?? "";
refreshP2PRemoteOptions();
});
const unsubscribeLayoutReady = eventHub.onEvent(EVENT_LAYOUT_READY, () => {
refreshP2PRemoteOptions();
void requestServerStatus();
});
fireAndForget(async () => {
await delay(100);
refreshP2PRemoteOptions();
await requestServerStatus();
});
return () => {
unsubscribe();
unsubscribeReplicatorStatus();
unsubscribeReplicatorProgress();
unsubscribeSettings();
unsubscribeLayoutReady();
};
});
function getAcceptanceStatus(peer: P2PServerInfo["knownAdvertisements"][number]) {
if (peer.isTemporaryAccepted === true) return "ACCEPTED (in session)";
if (peer.isAccepted === true) return "ACCEPTED";
if (peer.isTemporaryAccepted === false) return "DENIED (in session)";
if (peer.isAccepted === false) return "DENIED";
return "NEW";
}
function getAcceptanceStatusClass(peer: P2PServerInfo["knownAdvertisements"][number]) {
if (peer.isTemporaryAccepted === true || peer.isAccepted === true) return "accepted";
if (peer.isTemporaryAccepted === false || peer.isAccepted === false) return "denied";
return "unknown";
}
function openConnectionSettings() {
eventHub.emitEvent(EVENT_REQUEST_OPEN_P2P_SETTINGS);
}
async function applyP2PActiveRemoteSelection(id: string) {
selectingP2PRemote = true;
try {
await core.services.setting.updateSettings((settings) => {
settings.P2P_ActiveRemoteConfigurationId = id;
if (id.trim() === "") {
return settings;
}
const activated = activateP2PRemoteConfiguration(settings, id);
return activated || settings;
}, true);
const latest = core.services.setting.currentSettings();
syncOnReplicationSetting = latest.P2P_SyncOnReplication ?? "";
refreshP2PRemoteOptions();
} finally {
selectingP2PRemote = false;
}
}
async function onP2PRemoteSelected(event: Event) {
const target = event.currentTarget as HTMLSelectElement;
const id = target.value;
selectedP2PRemoteConfigurationId = id;
await applyP2PActiveRemoteSelection(id);
}
async function createAndSelectP2PRemote() {
const setupManager = core.getModule(SetupManager);
const dialogManager = setupManager.dialogManager;
const currentSettings = core.services.setting.currentSettings();
const p2pConf = await dialogManager.openWithExplicitCancel(SetupRemoteP2P, currentSettings);
if (p2pConf === "cancelled" || typeof p2pConf !== "object" || !p2pConf) {
return;
}
const p2pSettings = p2pConf as Partial<P2PSyncSetting>;
const id = createRemoteConfigurationId();
const roomSuffix = extractP2PRoomSuffix(p2pSettings.P2P_roomID ?? "");
const name = roomSuffix ? `P2P Remote (${roomSuffix})` : "P2P Remote";
await core.services.setting.updateSettings((settings) => {
const merged = {
...settings,
...p2pSettings,
};
const uri = ConnectionStringParser.serialize({ type: "p2p", settings: merged });
settings.remoteConfigurations = {
...(settings.remoteConfigurations ?? {}),
[id]: {
id,
name,
uri,
isEncrypted: false,
},
};
settings.P2P_ActiveRemoteConfigurationId = id;
const activated = activateP2PRemoteConfiguration(settings, id);
return activated || settings;
}, true);
const latest = core.services.setting.currentSettings();
syncOnReplicationSetting = latest.P2P_SyncOnReplication ?? "";
refreshP2PRemoteOptions();
}
async function updateSelectedP2PRemote(partial: Partial<P2PSyncSetting>) {
const selectedId = core.services.setting.currentSettings().P2P_ActiveRemoteConfigurationId?.trim() ?? "";
if (selectedId === "") {
return;
}
await core.services.setting.updateSettings((settings) => {
const config = settings.remoteConfigurations?.[selectedId];
if (!config) {
return settings;
}
let parsed;
try {
parsed = ConnectionStringParser.parse(config.uri);
} catch {
return settings;
}
if (parsed.type !== "p2p") {
return settings;
}
const mergedP2P = {
...parsed.settings,
...partial,
};
const uri = ConnectionStringParser.serialize({
type: "p2p",
settings: {
...settings,
...mergedP2P,
},
});
settings.remoteConfigurations = {
...(settings.remoteConfigurations ?? {}),
[selectedId]: {
...config,
uri,
isEncrypted: false,
},
};
Object.assign(settings, partial);
const activated = activateP2PRemoteConfiguration(settings, selectedId);
return activated || settings;
}, true);
syncOnReplicationSetting = core.services.setting.currentSettings()?.P2P_SyncOnReplication ?? "";
}
async function makeDecision(
peer: P2PServerInfo["knownAdvertisements"][number],
decision: boolean,
isTemporary: boolean
) {
decidingPeerId = peer.peerId;
try {
await liveSyncReplicator.makeDecision({
peerId: peer.peerId,
name: peer.name,
decision,
isTemporary,
});
await requestServerStatus();
} finally {
decidingPeerId = null;
}
}
async function revokeDecision(peer: P2PServerInfo["knownAdvertisements"][number]) {
decidingPeerId = peer.peerId;
try {
await liveSyncReplicator.revokeDecision({
peerId: peer.peerId,
name: peer.name,
});
await requestServerStatus();
} finally {
decidingPeerId = null;
}
}
async function startReplication(peer: P2PServerInfo["knownAdvertisements"][number]) {
replicatingPeerId = peer.peerId;
try {
const pullResult = await liveSyncReplicator.replicateFrom(peer.peerId, true);
if (pullResult?.ok) {
await liveSyncReplicator.requestSynchroniseToPeer(peer.peerId);
}
await requestServerStatus();
} finally {
replicatingPeerId = null;
}
}
function isAccepted(peer: P2PServerInfo["knownAdvertisements"][number]) {
return peer.isTemporaryAccepted === true || peer.isAccepted === true;
}
function isWatching(peerId: string) {
return replicatorInfo?.watchingPeers?.includes(peerId) ?? false;
}
function toggleWatch(peerId: string) {
if (!canEditP2PSettings()) {
return;
}
if (isWatching(peerId)) {
liveSyncReplicator.unwatchPeer(peerId);
} else {
liveSyncReplicator.watchPeer(peerId);
}
}
function isCommunicating(peerId: string) {
const to = replicatorInfo?.replicatingTo ?? [];
const from = replicatorInfo?.replicatingFrom ?? [];
const isLiveCommunicating = to.includes(peerId) || from.includes(peerId);
const isHeldCommunicating = (communicatingUntil[peerId] ?? 0) > Date.now();
return isLiveCommunicating || isHeldCommunicating;
}
function isSyncTarget(peerName: string) {
return syncOnReplicationSetting
.split(",")
.map((e) => e.trim())
.filter((e) => e)
.includes(peerName);
}
async function toggleSyncTarget(peer: P2PServerInfo["knownAdvertisements"][number]) {
if (!canEditP2PSettings()) {
return;
}
const currentValue = core.services.setting.currentSettings()?.P2P_SyncOnReplication ?? "";
const newValue = isSyncTarget(peer.name)
? removeFromList(peer.name, currentValue)
: addToList(peer.name, currentValue);
await updateSelectedP2PRemote({ P2P_SyncOnReplication: newValue });
}
</script>
<div class="p2p-container">
<div class="pane-header">
<h2>P2P Status</h2>
<div class="pane-header-actions">
<div class="remote-picker-wrap">
<select
class="remote-picker"
value={selectedP2PRemoteConfigurationId}
onchange={onP2PRemoteSelected}
disabled={selectingP2PRemote}
aria-label="Select active P2P remote"
title="Select active P2P remote"
>
{#if p2pRemoteOptions.length === 0}
<option value="">Select P2P remote...</option>
{/if}
{#each p2pRemoteOptions as option}
<option value={option.id}>
{option.name}{option.roomSuffix ? ` (${option.roomSuffix})` : ""}
</option>
{/each}
</select>
<button
class="icon-button"
onclick={() => createAndSelectP2PRemote()}
title="Create P2P remote"
aria-label="Create P2P remote"
>
+
</button>
</div>
<button
class="icon-button"
onclick={openConnectionSettings}
title="Open P2P Setup..."
aria-label="Open P2P Setup..."
>
</button>
</div>
</div>
{#if !canEditP2PSettings()}
<p class="warning-line">Please select an active P2P remote configuration to change P2P sync targets.</p>
{/if}
<P2PServerStatusCard {liveSyncReplicator} {core} />
<div class="peers-section">
<div class="peers-header">
<h3>Detected Peers</h3>
<button class="refresh" onclick={requestServerStatus}>Refresh</button>
</div>
{#if serverInfo && serverInfo.knownAdvertisements.length > 0}
<div class="peers-list">
{#each serverInfo.knownAdvertisements as peer (peer.peerId)}
<div class="peer-item">
<div class="peer-info">
<div class="peer-name">
{peer.name} :
<span class="peer-id-mini" title={peer.peerId}>({peer.peerId.slice(0, 8)})</span>
{#if isCommunicating(peer.peerId)}
<span class="comm-icon" title="Communicating" aria-label="Communicating">📡</span>
{/if}
</div>
<div class="peer-meta">
<span class="badge">{peer.platform}</span>
</div>
</div>
<div class="peer-actions">
{#if isAccepted(peer)}
<div class="decision-row accepted-row">
<span class="badge status-chip {getAcceptanceStatusClass(peer)}">
{getAcceptanceStatus(peer)}
</span>
<button
class="emoji-button"
disabled={replicatingPeerId !== null}
title={replicatingPeerId === peer.peerId ? "Replicating..." : "Replicate now"}
aria-label={replicatingPeerId === peer.peerId ? "Replicating" : "Replicate now"}
onclick={() => startReplication(peer)}
>
{replicatingPeerId === peer.peerId ? "⏳" : "🔄"}
</button>
<button
class="action-button"
disabled={decidingPeerId !== null}
onclick={() => revokeDecision(peer)}
>
Revoke
</button>
</div>
<div class="decision-row watch-row">
<span class="decision-label">WATCH</span>
<button
class="emoji-button {isWatching(peer.peerId) ? 'is-watching' : ''}"
title={isWatching(peer.peerId)
? "Watching this peer \u2014 click to stop"
: "Watch this peer's changes"}
aria-label={isWatching(peer.peerId) ? "Stop watching" : "Watch peer"}
disabled={!canEditP2PSettings()}
onclick={() => toggleWatch(peer.peerId)}
>
{isWatching(peer.peerId) ? "🔔" : "🔕"}
</button>
</div>
<div class="decision-row watch-row">
<span class="decision-label">SYNC</span>
<button
class="emoji-button {isSyncTarget(peer.name) ? 'is-watching' : ''}"
title={isSyncTarget(peer.name)
? "Sync target \u2014 click to remove"
: "Set as sync target"}
aria-label={isSyncTarget(peer.name) ? "Remove sync target" : "Set sync target"}
disabled={!canEditP2PSettings()}
onclick={() => toggleSyncTarget(peer)}
>
{isSyncTarget(peer.name) ? "🔗" : "⛓️‍💥"}
</button>
</div>
{:else}
<div class="decision-status">
<span class="badge status-chip {getAcceptanceStatusClass(peer)}">
{getAcceptanceStatus(peer)}
</span>
</div>
<div class="decision-row">
<span class="decision-label">PERMANENT</span>
<button
class="emoji-button"
title="Allow permanently"
aria-label="Allow permanently"
disabled={decidingPeerId !== null}
onclick={() => makeDecision(peer, true, false)}
>
</button>
<button
class="emoji-button mod-warning"
title="Deny permanently"
aria-label="Deny permanently"
disabled={decidingPeerId !== null}
onclick={() => makeDecision(peer, false, false)}
>
🚫
</button>
</div>
<div class="decision-row">
<span class="decision-label">SESSION</span>
<button
class="emoji-button"
title="Allow in session"
aria-label="Allow in session"
disabled={decidingPeerId !== null}
onclick={() => makeDecision(peer, true, true)}
>
</button>
<button
class="emoji-button mod-warning"
title="Deny in session"
aria-label="Deny in session"
disabled={decidingPeerId !== null}
onclick={() => makeDecision(peer, false, true)}
>
🚫
</button>
</div>
{/if}
{#if !isAccepted(peer) && (peer.isAccepted !== undefined || peer.isTemporaryAccepted !== undefined)}
<button
class="action-button revoke-inline"
disabled={decidingPeerId !== null}
onclick={() => revokeDecision(peer)}
>
Revoke
</button>
{/if}
</div>
</div>
{/each}
</div>
{:else if serverInfo}
<p class="no-peers">No devices available. Waiting for other devices to connect...</p>
{:else}
<p class="no-peers">Fetching status...</p>
{/if}
</div>
</div>
<style>
.p2p-container {
display: flex;
flex-direction: column;
gap: 1rem;
}
.peers-section {
border: 1px solid var(--divider-color);
border-radius: 0.5rem;
padding: 1rem;
}
.pane-header {
display: flex;
justify-content: space-between;
align-items: center;
flex-wrap: wrap;
gap: 0.5rem;
}
.pane-header-actions {
display: flex;
align-items: center;
gap: 0.4rem;
min-width: 0;
}
.remote-picker-wrap {
display: inline-flex;
gap: 0.3rem;
align-items: center;
min-width: 0;
}
.remote-picker {
max-width: 10rem;
min-width: 1em;
flex-shrink: 1;
height: 1.9rem;
border: 1px solid var(--divider-color);
border-radius: 0.4rem;
background-color: var(--interactive-normal);
color: var(--text-normal);
padding: 0 0.45rem;
}
.warning-line {
margin: -0.2rem 0 0;
font-size: 0.82rem;
color: var(--text-warning);
}
.pane-header h2 {
margin: 0;
font-size: 1.1rem;
font-weight: 700;
white-space: nowrap;
}
.icon-button {
width: 1.9rem;
height: 1.9rem;
display: inline-flex;
align-items: center;
justify-content: center;
font-size: 1rem;
line-height: 1;
border: 1px solid var(--divider-color);
border-radius: 0.4rem;
background-color: var(--interactive-normal);
color: var(--text-normal);
cursor: pointer;
flex-shrink: 0;
}
.icon-button:hover {
background-color: var(--interactive-hover);
}
.peers-header {
display: flex;
flex-wrap: wrap;
justify-content: space-between;
align-items: center;
gap: 0.5rem;
margin-bottom: 0.75rem;
}
h3 {
margin: 0;
font-weight: 600;
font-size: 1rem;
}
.refresh {
font-size: 0.8rem;
padding: 0.2rem 0.5rem;
border: 1px solid var(--divider-color);
border-radius: 0.3rem;
background-color: var(--interactive-normal);
color: var(--text-normal);
cursor: pointer;
}
.refresh:hover {
background-color: var(--interactive-hover);
}
.peers-list {
display: flex;
flex-direction: column;
gap: 0.5rem;
}
.peer-item {
display: flex;
flex-direction: column;
align-items: stretch;
gap: 0.75rem;
padding: 0.75rem;
background-color: var(--background-secondary);
border: 1px solid var(--divider-color);
border-radius: 0.4rem;
}
.peer-info {
flex: 1;
min-width: 0;
}
.peer-name {
font-weight: 600;
margin-bottom: 0.25rem;
font-size: 0.9rem;
display: flex;
align-items: center;
gap: 0.35rem;
}
.peer-meta {
display: flex;
gap: 0.5rem;
font-size: 0.8rem;
flex-wrap: wrap;
}
.badge {
background-color: var(--background-tertiary);
padding: 0.1rem 0.4rem;
border-radius: 0.25rem;
}
.status-chip {
font-weight: 600;
}
.status-chip.accepted {
background-color: var(--background-modifier-success);
color: var(--text-normal);
}
.status-chip.denied {
background-color: var(--background-modifier-error);
color: var(--text-normal);
}
.status-chip.unknown {
background-color: var(--background-modifier-border);
color: var(--text-muted);
}
.peer-id-mini {
font-family: monospace;
color: var(--text-muted);
font-size: 0.75rem;
}
.comm-icon {
font-size: 0.8rem;
line-height: 1;
animation: pulse-comm 1.2s ease-in-out infinite;
}
@keyframes pulse-comm {
0% {
opacity: 0.55;
transform: scale(0.95);
}
50% {
opacity: 1;
transform: scale(1);
}
100% {
opacity: 0.55;
transform: scale(0.95);
}
}
.peer-actions {
display: flex;
flex-direction: column;
gap: 0.35rem;
width: 100%;
min-width: 0;
}
.decision-status {
display: flex;
justify-content: flex-start;
}
.decision-row {
display: grid;
grid-template-columns: 1fr auto auto;
align-items: center;
gap: 0.35rem;
}
.accepted-row {
grid-template-columns: 1fr auto auto;
}
.decision-label {
font-size: 0.7rem;
font-weight: 700;
color: var(--text-muted);
letter-spacing: 0.03em;
}
.action-button {
font-size: 0.75rem;
padding: 0.2rem 0.45rem;
border: 1px solid var(--divider-color);
border-radius: 0.3rem;
background-color: var(--interactive-normal);
color: var(--text-normal);
cursor: pointer;
width: auto;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.emoji-button {
width: 2rem;
height: 1.7rem;
display: inline-flex;
align-items: center;
justify-content: center;
border: 1px solid var(--divider-color);
border-radius: 0.3rem;
background-color: var(--interactive-normal);
cursor: pointer;
padding: 0;
line-height: 1;
}
.emoji-button.mod-warning {
background-color: var(--background-modifier-error);
}
.emoji-button.is-watching {
background-color: var(--interactive-accent);
color: var(--text-on-accent);
border-color: var(--interactive-accent);
}
.emoji-button:hover:not(:disabled) {
background-color: var(--interactive-hover);
}
.emoji-button.mod-warning:hover:not(:disabled) {
filter: brightness(0.95);
}
.watch-row {
margin-top: 0.25rem;
}
.action-button:hover:not(:disabled) {
background-color: var(--interactive-hover);
}
.action-button.mod-warning {
background-color: var(--background-modifier-error);
}
.action-button:disabled {
opacity: 0.6;
cursor: not-allowed;
}
.emoji-button:disabled {
opacity: 0.6;
cursor: not-allowed;
}
.revoke-inline {
justify-self: start;
}
.no-peers {
text-align: center;
color: var(--text-muted);
font-size: 0.9rem;
padding: 1rem;
}
</style>

Some files were not shown because too many files have changed in this diff Show More